Job description

Job Description We are seeking a highly skilled Lead Data Engineer to join our dynamic data team. The ideal candidate has extensive experience designing, building, and optimizing modern data pipelines and cloud‑based data platforms. In this role, you will help shape scalable data infrastructure that supports analytics, machine learning, and business‑critical applications across the organization. What you will do: • Design, develop, and maintain scalable data pipelines to ingest, transform, and deliver large and diverse datasets. • Write efficient, reusable SQL and/or Python for data ingestion, transformation, and integration. • Optimize data workflows for performance, reliability, security, and cost‑efficiency in cloud environments. • Collaborate with data scientists, analysts, application teams, and business stakeholders to understand data requirements and deliver technical solutions. • Implement data governance and security practices across pipeline components and data assets. • Monitor data flows, troubleshoot issues, and ensure data quality, availability, and integrity. • Integrate data from various internal and external sources into centralized data platforms. • Provide technical leadership, mentor junior engineers, and support knowledge‑sharing within the team. • Evaluate emerging tools and architectures, recommending improvements to existing systems and processes. • Conduct performance tuning and optimization across ingestion, transformation, and storage layers. What you should have: • Bachelor’s degree in Computer Science, Engineering, or related field, or equivalent experience. • 12–14+ years of experience in data or related engineering roles, with 8+ years focused on data engineering. • Advanced proficiency in SQL and/or Python for data processing, transformation, and automation. • Hands-on experience building and optimizing data pipelines and integration workflows. • Strong understanding of data modeling, performance optimization, and query tuning. • Experience working with cloud platforms and their data ecosystem. • Knowledge of ETL/ELT concepts, orchestration patterns, and automated data workflows. • Understanding of data governance, data security, and handling sensitive or regulated data (e.g., PII). • Experience with version control and CI/CD practices. • Strong problem-solving abilities with experience in complex or unstructured data environments. • Strong analytical and troubleshooting skills, including root-cause analysis. Preferred qualifications: • Experience with distributed data processing or declarative pipeline frameworks. • Familiarity with cloud-native functions and workflow orchestration tools. • Experience with data governance frameworks, metadata management, or cataloging solutions. • Knowledge of columnar storage and big-data file formats. • Familiarity with modern table formats such as Parquet, Delta, or Iceberg. Required Soft Skills: • Strong leadership and mentorship abilities, with a focus on elevating team capability and fostering knowledge sharing. • Strategic thinking and the ability to align engineering work with broader business objectives. • Effective communication skills, including the ability to translate technical concepts for non‑technical audiences. • Collaboration-oriented mindset, thriving in cross-functional environments with product, analytics, and engineering partners. • Adaptability and resilience in navigating ambiguity, shifting priorities, or evolving technical landscapes. • Proactive initiative, consistently identifying opportunities for improvement and driving solutions forward. • Commitment to quality and craft, with a strong sense of ownership over system performance and reliability. • Growth mindset, continuously seeking new knowledge, tools, and approaches to improve team outcomes. AIR offers attractive total compensation packages designed to recognize and reward performance at the individual, team and company levels. We start with a competitive base salary and add bonus opportunities and benefit choices topped off with: • Expected pay rate of $ $135,000 - $160,000 plus annual bonus . You will be provided a pay rate that is in line with your skills and experience as they relate to the requirements of the job. • 25% Discounted Rent at AIR communities (certain exclusions apply, and an application is required). • Medical, dental, vision, and life insurance options available. HSA with company sponsored contributions, FSA (traditional and dependent care FSA). • Short and long-term disability paid by the company. • 401(K) plus company contributions. • Paid Vacation, 14 Paid Holidays, and Paid Sick Time. • Tuition Assistance Program. • Commuter Benefits and Pet-Insurance. • Up to 100% Reimbursement on Professional Certification/Licensing for Job Related Curriculums. • Paid parental leave of up to 16 weeks, following the birth or adoption of a child. • Consumer discounts from various products and services including travel planning, wireless phones, health and fitness, home appliances, and more. • 15 hours of paid time annually to give-back to your community. • Opportunities for professional development and career growth including realistic job previews, in person and virtual leadership training, and performance improvement courses. Application Deadline: The initial deadline for applications is 6/9/2026. Applications submitted after this date may still be considered on a rolling basis until the position/program is filled. We encourage all interested candidates to apply as early as possible. If a final decision has not been made by the initial deadline, subsequent applications will be reviewed as they are received. Candidates extended an offer will be required to undergo pre-employment screening that includes checking references, a four-panel drug screening that excludes marijuana testing, employment verification & criminal background check. AIR values diversity and provides equal employment opportunities to all employees and applicants for employment. AIR prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.

Requirements

Must-haves

Design, develop, and maintain scalable data pipelines to ingest, transform, and deliver large and diverse datasets.
Write efficient, reusable SQL and/or Python for data ingestion, transformation, and integration.
Optimize data workflows for performance, reliability, security, and cost‑efficiency in cloud environments.
Collaborate with data scientists, analysts, application teams, and business stakeholders to understand data requirements and deliver technical solutions.
Implement data governance and security practices across pipeline components and data assets.
Monitor data flows, troubleshoot issues, and ensure data quality, availability, and integrity.
Integrate data from various internal and external sources into centralized data platforms.
Provide technical leadership, mentor junior engineers, and support knowledge‑sharing within the team.
Evaluate emerging tools and architectures, recommending improvements to existing systems and processes.
Conduct performance tuning and optimization across ingestion, transformation, and storage layers.
Bachelor’s degree in Computer Science, Engineering, or related field, or equivalent experience.
12–14+ years of experience in data or related engineering roles, with 8+ years focused on data engineering.
Advanced proficiency in SQL and/or Python for data processing, transformation, and automation.
Hands-on experience building and optimizing data pipelines and integration workflows.
Strong understanding of data modeling, performance optimization, and query tuning.
Experience working with cloud platforms and their data ecosystem.
Knowledge of ETL/ELT concepts, orchestration patterns, and automated data workflows.
Understanding of data governance, data security, and handling sensitive or regulated data (e.g., PII).

Nice-to-haves

Experience with distributed data processing or declarative pipeline frameworks.
Familiarity with cloud-native functions and workflow orchestration tools.
Experience with data governance frameworks, metadata management, or cataloging solutions.
Knowledge of columnar storage and big-data file formats.
Familiarity with modern table formats such as Parquet, Delta, or Iceberg.
Expected pay rate of $ $135,000 - $160,000 plus annual bonus . You will be provided a pay rate that is in line with your skills and experience as they relate to the requirements of the job.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

machine learningsqlpythonci/cd

Platform: smartrecruiters
Effort: medium

Benefits

Benefits detected in the listing.

Dental

Vision

401(K)

Parental Leave

Professional Development