Senior Platform Engineer (Databricks)
📍 Job Overview
- Job Title: Senior Platform Engineer (Databricks)
- Company: Live Nation
- Location: Work From Home - New York
- Job Type: Full-Time
- Category: DevOps Engineer
- Date Posted: 2025-06-25
- Experience Level: 5-10 years
- Remote Status: Remote OK
🚀 Role Summary
- Key Responsibilities: Enhance and improve the performance and reliability of existing Data Platform Services. Design and build scalable, reliable, and secure onboarding and automation capabilities for the Core Data Platform services.
- Key Skills: Python, PySpark, Data Lakes, Data Warehouses, Git, CI/CD, Data Ingestion, ANSI SQL, Spark SQL, Airflow, Databricks, Hadoop, AWS, GCP, Azure, Data Governance, Agile Development
💻 Primary Responsibilities
-
📝 Enhancement Note: This role is not focused on ETL/ELT development. Instead, it requires strong Python/PySpark coding skills to contribute to the enhancements and continuous improvement of the existing Data Platform Services.
-
📝 Enhancement Note: The Senior Platform Engineer will design and develop scalable data platform and integration solutions for various data sources leveraging Databricks Unified Platform. They will also ensure data quality, security, and compliance with relevant regulations.
-
📝 Enhancement Note: This role involves designing and building platform capabilities to support onboarding and operation effectiveness. The Senior Platform Engineer will also participate in on-call rotations/Pagerduty for data platform support.
-
📝 Enhancement Note: The Senior Platform Engineer will collaborate with cross-functional teams to deliver data solutions and provide technical support and guidance to team members. They will stay up-to-date with the latest data engineering technologies and trends.
🎓 Skills & Qualifications
Education: Bachelor's degree in Computer Science, Engineering, or a related field. Relevant work experience may be considered in lieu of a degree.
Experience: 5-10 years of experience in data engineering, with a strong focus on platform engineering and data management.
Required Skills:
- Strong Python/PySpark coding skills
- Good understanding of Data Lakes and Data Warehouses
- Experience using version control systems such as Git and CI/CD workflows
- Design and development of data ingestion services that are highly performant, reliable, and scalable
- Decent expertise using ANSI SQL and Spark SQL
- Workflow automation, orchestration using Airflow or equivalent tools stack
- Hands-on working knowledge of at least one of the following: Databricks, Hadoop, and related stacks
- Experience with cloud services from any of the following: Amazon AWS, Google GCP, or Microsoft Azure (preferably AWS)
- Experience with streaming (Kafka) and batch-based data sources
- Diverse data sources and data formats (xml, json, yaml, parquet, avro, delta) and respective use cases
- Agile development methodologies using the Atlassian suite: Jira, Confluence
Preferred Skills:
- Experience with data visualization tools
- Understanding of the nuances around time zones, geo, various data formats, data types across different storage systems
- Familiarity with the latest data engineering technologies and trends
📊 Web Portfolio & Project Requirements
Portfolio Essentials:
-
📝 Enhancement Note: As this is a platform engineering role, the portfolio should focus on showcasing the candidate's ability to design, build, and maintain scalable, reliable, and secure data platforms. Include examples of data ingestion services, data governance, and data quality initiatives.
-
📝 Enhancement Note: The portfolio should demonstrate the candidate's ability to work with various data sources, data formats, and cloud services. Include case studies that highlight the candidate's problem-solving skills and ability to optimize data platform performance.
Technical Documentation:
- Detailed documentation of data platform architecture, data flows, and data transformations
- Code comments and inline documentation demonstrating best practices
- Version control and CI/CD pipeline documentation
💵 Compensation & Benefits
Salary Range: $128,000.00 USD - $160,000.00 USD per year
Benefits:
- Medical, Dental, and Vision benefits for you and your family, including Flexible Spending Accounts (FSA) and Health Savings Accounts (HSAs)
- Paid time off policy including holidays, sick time, and day off for your birthday, free concert tickets
- 401(k) program with company match, Stock Program Reimbursement
- New parent programs and support including caregiver leave and baby bonus, infertility support
- Tuition reimbursement, student loan repayment, internal growth, and development programs & trainings
- Volunteer time off, crowdfunding network
🎯 Team & Company Context
Company Culture:
-
📝 Enhancement Note: Live Nation Entertainment is the world's leading live entertainment company, comprised of global market leaders: Ticketmaster, Live Nation Concerts, and Live Nation Media & Sponsorship. This role is part of the Core Data Services team, which is at the center of Data and Analytics initiatives.
-
Industry: Live Nation operates in the live entertainment industry, with a focus on event ticketing, live concerts, and music marketing. This role will be working with data from various sources across the company.
-
Company Size: Live Nation Entertainment has over 55,000 employees worldwide. This role will be part of a large, cross-functional team working on data platforms and services.
-
Founded: Live Nation Entertainment was founded in 2005, following the merger of Clear Channel Entertainment and Live Nation. The company has a rich history in the live entertainment industry and a strong focus on innovation and growth.
Team Structure:
-
The Core Data Services team consists of Platform Engineering, Data Engineering, Data Enablement, and Operations teams. This role will be part of the Platform Engineering team, working closely with other teams to deliver data solutions and services.
-
📝 Enhancement Note: The team follows Agile development methodologies using the Atlassian suite: Jira, Confluence. The team structure is designed to maximize collaboration, innovation, and efficiency.
Development Methodology:
-
The team follows Agile development methodologies, with a focus on continuous integration, continuous deployment, and continuous improvement. The team uses Jira for project management and Confluence for knowledge sharing and collaboration.
-
📝 Enhancement Note: The team emphasizes code reviews, pair programming, and regular retrospectives to ensure high-quality code, efficient processes, and continuous learning.
Company Website: https://www.livenationentertainment.com/
📈 Career & Growth Analysis
Web Technology Career Level: Senior Platform Engineer (Databricks) - This role is a senior-level position within the data engineering and platform engineering career path. It requires a deep understanding of data platforms, data management, and data governance. The role involves designing, building, and maintaining scalable, reliable, and secure data platforms, as well as collaborating with cross-functional teams to deliver data solutions and services.
Reporting Structure: This role reports directly to the Manager of Platform Engineering within the Core Data Services team. The team is part of the broader Data and Analytics organization, working closely with data engineering, data science, and data analysis teams.
Technical Impact: The Senior Platform Engineer (Databricks) has a significant impact on the data platforms and services used across the company. Their work ensures that data is accurate, complete, consistent, timely, and accessible to users across the organization. Their decisions and implementations directly influence the performance, reliability, and scalability of the data platforms and services.
Growth Opportunities:
-
📝 Enhancement Note: Live Nation Entertainment offers a range of growth opportunities for technical professionals. These include technical skill development, leadership roles, and architecture decision-making.
-
Growth Opportunity 1 - Technical Skill Development: Live Nation Entertainment encourages its employees to develop their technical skills and specialize in emerging technologies. The company offers tuition reimbursement, student loan repayment, and internal training programs to support this growth.
-
Growth Opportunity 2 - Technical Leadership: As a senior-level role, the Senior Platform Engineer (Databricks) has the opportunity to mentor junior team members, lead projects, and influence the technical direction of the team. This role can lead to technical leadership positions within the team or the broader organization.
-
Growth Opportunity 3 - Architecture Decision-Making: The Senior Platform Engineer (Databricks) is involved in critical architecture decisions that impact the data platforms and services used across the company. This role offers the opportunity to gain experience in architecture decision-making and to influence the technical direction of the organization.
🌐 Work Environment
Office Type: Live Nation Entertainment has a hybrid work environment, with a combination of on-site and remote work options. This role is currently offered as a remote position, with the possibility of on-site work in the future.
Office Location(s): The company has offices worldwide, with a strong presence in the United States. This role is currently offered as a remote position, with the possibility of on-site work in the future.
Workspace Context:
-
📝 Enhancement Note: As a remote position, the Senior Platform Engineer (Databricks) will work from a home office or co-working space. The company provides remote work equipment and supports a comfortable and productive work environment.
-
Workspace Aspect 1 - Collaboration: The role involves regular collaboration with cross-functional teams, including data engineering, data science, and data analysis teams. The team uses communication and collaboration tools such as Slack, Microsoft Teams, and Google Workspace to facilitate remote work.
-
Workspace Aspect 2 - Work Arrangement: The role follows a standard full-time work arrangement, with a flexible schedule to accommodate meetings and collaboration with team members in different time zones. The company offers a competitive benefits package, including paid time off, health insurance, and retirement plans.
-
Workspace Aspect 3 - Work-Life Balance: Live Nation Entertainment is committed to supporting work-life balance for its employees. The company offers flexible work arrangements, including part-time work and job sharing, as well as remote work options.
Work Schedule: The role follows a standard full-time work schedule, with a flexible schedule to accommodate meetings and collaboration with team members in different time zones. The company offers a competitive benefits package, including paid time off, health insurance, and retirement plans.
📄 Application & Technical Interview Process
Interview Process:
- 📝 Enhancement Note: The interview process for this role is designed to assess the candidate's technical skills, problem-solving abilities, and cultural fit with the team. The process typically includes the following steps:
- Phone Screen: A brief phone call to discuss the candidate's qualifications, experience, and career goals.
- Technical Phone Interview: A deeper dive into the candidate's technical skills, with a focus on Python/PySpark coding, data management, and data governance.
- On-Site Interview: A visit to the company's headquarters or a local office to meet with team members, discuss the role in more detail, and complete a technical assessment.
- Final Decision: A final decision is made based on the candidate's technical skills, cultural fit, and alignment with the team's goals and objectives.
Portfolio Review Tips:
-
📝 Enhancement Note: The portfolio should showcase the candidate's ability to design, build, and maintain scalable, reliable, and secure data platforms. Include examples of data ingestion services, data governance, and data quality initiatives.
-
📝 Enhancement Note: The portfolio should demonstrate the candidate's ability to work with various data sources, data formats, and cloud services. Include case studies that highlight the candidate's problem-solving skills and ability to optimize data platform performance.
Technical Challenge Preparation:
-
📝 Enhancement Note: The technical interview for this role is designed to assess the candidate's ability to design, build, and maintain scalable, reliable, and secure data platforms. The interview may include coding challenges, architecture design exercises, and problem-solving scenarios.
-
📝 Enhancement Note: The interview may also include a review of the candidate's portfolio, with a focus on the candidate's ability to work with various data sources, data formats, and cloud services. The interview may include questions about the candidate's approach to data management, data governance, and data quality.
🛠 Technology Stack & Web Infrastructure
Frontend Technologies: Not applicable to this role.
Backend & Server Technologies:
-
📝 Enhancement Note: The Senior Platform Engineer (Databricks) works with various backend and server technologies, with a focus on data management, data governance, and data quality.
-
Backend Technology 1 - Databricks: The role involves working with Databricks, a unified data analytics platform that combines data science, engineering, and business analytics. The Senior Platform Engineer (Databricks) will use Databricks to design, build, and maintain scalable, reliable, and secure data platforms.
-
Backend Technology 2 - Hadoop: The role may involve working with Hadoop, an open-source framework for distributed processing and storage of large data sets on computer clusters. The Senior Platform Engineer (Databricks) may use Hadoop to process and store large data sets, as well as to integrate with other data platforms and services.
-
Backend Technology 3 - AWS: The role involves working with Amazon Web Services (AWS), a comprehensive, evolving cloud computing platform offered by Amazon. The Senior Platform Engineer (Databricks) will use AWS to design, build, and maintain scalable, reliable, and secure data platforms, as well as to integrate with other data platforms and services.
Development & DevOps Tools:
-
📝 Enhancement Note: The Senior Platform Engineer (Databricks) uses various development and DevOps tools to design, build, and maintain scalable, reliable, and secure data platforms. These tools may include the following:
-
Development Tool 1 - Git: The role involves using Git, a distributed version control system that tracks changes in source code during software development. The Senior Platform Engineer (Databricks) will use Git to manage and version control the codebase for the data platforms and services.
-
DevOps Tool 2 - CI/CD Pipelines: The role involves using CI/CD pipelines to automate the software delivery process. The Senior Platform Engineer (Databricks) will use CI/CD pipelines to ensure that the data platforms and services are built, tested, and deployed efficiently and reliably.
-
Monitoring Tool 3 - Prometheus/Grafana: The role may involve using Prometheus and Grafana, open-source monitoring and visualization tools. The Senior Platform Engineer (Databricks) may use Prometheus and Grafana to monitor the performance and health of the data platforms and services, as well as to identify and troubleshoot issues.
👥 Team Culture & Values
Web Development Values:
-
📝 Enhancement Note: Live Nation Entertainment values innovation, collaboration, and continuous learning. The company encourages its employees to take risks, learn from their mistakes, and continuously improve their skills and knowledge.
-
Web Development Value 1 - Innovation: Live Nation Entertainment encourages its employees to think creatively and challenge the status quo. The company values innovation and rewards employees who drive change and improvement.
-
Web Development Value 2 - Collaboration: Live Nation Entertainment fosters a culture of collaboration and teamwork. The company values cross-functional collaboration and encourages employees to work together to achieve common goals and objectives.
-
Web Development Value 3 - Continuous Learning: Live Nation Entertainment encourages its employees to continuously learn and develop their skills. The company offers training and development opportunities to help employees grow and advance in their careers.
-
Web Development Value 4 - User-Centric Design: Live Nation Entertainment values user-centered design and user experience. The company encourages its employees to focus on the user and to design and build products and services that meet the needs of the user.
Collaboration Style:
-
📝 Enhancement Note: Live Nation Entertainment fosters a culture of collaboration and teamwork. The company values cross-functional collaboration and encourages employees to work together to achieve common goals and objectives.
-
Collaboration Approach 1 - Cross-Functional Integration: Live Nation Entertainment encourages cross-functional collaboration between different teams and departments. The company values the integration of different perspectives and expertise in the problem-solving process.
-
Collaboration Approach 2 - Code Review Culture: Live Nation Entertainment values code reviews and peer programming as essential aspects of the software development process. The company encourages employees to review each other's code and to provide feedback and guidance.
-
Collaboration Approach 3 - Knowledge Sharing: Live Nation Entertainment encourages knowledge sharing and learning from one another. The company values the sharing of expertise and the collective advancement of knowledge and skills.
⚡ Challenges & Growth Opportunities
Technical Challenges:
-
📝 Enhancement Note: The Senior Platform Engineer (Databricks) faces several technical challenges in designing, building, and maintaining scalable, reliable, and secure data platforms. These challenges may include the following:
-
Web Development Challenge 1 - Data Ingestion: The Senior Platform Engineer (Databricks) must design and build data ingestion services that are highly performant, reliable, and scalable. This challenge involves working with various data sources, data formats, and cloud services to ensure that data is ingested efficiently and accurately.
-
Web Development Challenge 2 - Data Governance: The Senior Platform Engineer (Databricks) must ensure that data is accurate, complete, consistent, timely, and accessible to users across the organization. This challenge involves designing and implementing data governance policies and procedures to ensure data quality, security, and compliance with relevant regulations.
-
Web Development Challenge 3 - Data Quality: The Senior Platform Engineer (Databricks) must ensure that data is accurate, complete, consistent, timely, and accessible to users across the organization. This challenge involves designing and implementing data quality initiatives to ensure that data is accurate, complete, and reliable.
-
Web Development Challenge 4 - Performance Optimization: The Senior Platform Engineer (Databricks) must optimize the performance of the data platforms and services. This challenge involves identifying and addressing performance bottlenecks, as well as designing and implementing performance optimization strategies.
Learning & Development Opportunities:
-
📝 Enhancement Note: Live Nation Entertainment offers a range of learning and development opportunities for technical professionals. These opportunities may include the following:
-
Learning Opportunity 1 - Technical Skill Development: Live Nation Entertainment encourages its employees to develop their technical skills and specialize in emerging technologies. The company offers tuition reimbursement, student loan repayment, and internal training programs to support this growth.
-
Learning Opportunity 2 - Conference Attendance: Live Nation Entertainment supports its employees' attendance at industry conferences and events. The company values the sharing of knowledge and expertise with other professionals in the field.
-
Learning Opportunity 3 - Technical Mentorship: Live Nation Entertainment offers technical mentorship opportunities for employees at all levels. The company values the sharing of knowledge and expertise between team members and encourages employees to learn from one another.
💡 Interview Preparation
Technical Questions:
-
📝 Enhancement Note: The technical interview for this role is designed to assess the candidate's ability to design, build, and maintain scalable, reliable, and secure data platforms. The interview may include the following questions:
-
Technical Question 1 - Data Ingestion: How would you design and build a data ingestion service that is highly performant, reliable, and scalable? What technologies and tools would you use to ensure that data is ingested efficiently and accurately?
-
Technical Question 2 - Data Governance: How would you ensure that data is accurate, complete, consistent, timely, and accessible to users across the organization? What data governance policies and procedures would you implement to ensure data quality, security, and compliance with relevant regulations?
-
Technical Question 3 - Data Quality: How would you ensure that data is accurate, complete, consistent, timely, and accessible to users across the organization? What data quality initiatives would you design and implement to ensure that data is accurate, complete, and reliable?
-
Technical Question 4 - Performance Optimization: How would you optimize the performance of the data platforms and services? What strategies and techniques would you use to identify and address performance bottlenecks, as well as to design and implement performance optimization strategies?
Company & Culture Questions:
-
📝 Enhancement Note: The interview process for this role is designed to assess the candidate's cultural fit with the team and the company. The interview may include the following questions:
-
Technical Question 5 - Company Web Development Culture: How do you approach web development and data management in a team environment? How do you collaborate with cross-functional teams to deliver data solutions and services?
-
Technical Question 6 - Web Development Methodology: How do you approach web development and data management methodologies? How do you ensure that your development processes are efficient, effective, and aligned with the company's goals and objectives?
Portfolio Presentation Strategy:
-
📝 Enhancement Note: The portfolio review for this role is designed to assess the candidate's ability to design, build, and maintain scalable, reliable, and secure data platforms. The portfolio review may include the following strategies:
-
Portfolio Strategy 1 - Live Website Demonstration: The candidate should demonstrate their ability to design, build, and maintain scalable, reliable, and secure data platforms by presenting a live website or web application. The candidate should walk the interviewer through the platform, highlighting their design and implementation decisions.
-
Portfolio Strategy 2 - Code Explanation: The candidate should be prepared to explain their code and design decisions in detail. The candidate should be able to articulate their approach to data management, data governance, and data quality, as well as their strategies for performance optimization and scalability.
-
Portfolio Strategy 3 - User Experience Showcase: The candidate should demonstrate their understanding of user experience and user-centered design principles. The candidate should be able to articulate how their data platforms and services meet the needs of the user and how they ensure that the user experience is intuitive, engaging, and effective.
📌 Application Steps
To apply for this Senior Platform Engineer (Databricks) position at Live Nation Entertainment:
- 📝 Enhancement Note: Submit your application through the application link provided in the job listing.
- 📝 Enhancement Note: Customize your resume and portfolio to highlight your relevant experience and skills for this role. Include specific examples of your work with data platforms, data management, and data governance.
- 📝 Enhancement Note: Prepare for the technical interview by reviewing the technical questions and strategies outlined in this document. Practice your coding and problem-solving skills, and be prepared to articulate your approach to data management, data governance, and data quality.
- 📝 Enhancement Note: Research the company and the team to ensure that you are a good fit for the role and the organization. Be prepared to discuss your alignment with the company's values and culture.
- 📝 Enhancement Note: Follow up with the hiring manager or recruiter after the interview to express your interest in the role and to address any questions or concerns that may have arisen during the interview process.
📝 Enhancement Note: This enhanced job description includes AI-generated insights and web development/server administration industry-standard assumptions. All details should be verified directly with the hiring organization before making application decisions.
Application Requirements
Candidates must have strong Python/PySpark coding skills and a good understanding of Data Lakes and Data Warehouses. Experience with version control systems, cloud services, and data management best practices is essential.