Senior Data Engineer

What the role needs

• As a Senior Data Engineer, you will play a key role in building, optimizing, and maintaining scalable data architectures and pipelines that power critical healthcare data solutions.
• Develop and manage robust ETL/ELT pipelines for data ingestion, transformation, and integration from heterogeneous sources, including relational databases, APIs, file shares, and end-user-computing assets
• Develop Python and PySpark solutions for distributed data processing, automation, integration, and reusable transformation logic
• Optimize data platform performance and cloud consumption through SQL tuning, Spark job optimization, storage format selection, partitioning strategies, cluster configuration, and workload scheduling
• Build and optimize workflows on Azure Cloud services and Databricks - including medallion (bronze/silver/gold) layering and orchestrated pipelines - to support enterprise data needs
• Migrate business logic embedded in legacy and end-user-computing systems into documented, testable transformation logic over governed tables
• Define and implement data quality, validation, reconciliation, and parity testing practices to ensure migrated pipelines are accurate, reliable, auditable, and consistent with the systems they replace
• Establish data lineage, ownership, and cataloguing, and define documented rollup and allocation rules where source systems operate at differing grains
• Implement security and compliance measures for handling PHI/PII and confidential business data in accordance with HIPAA or similar regulations, including de-identification and masking for controlled sharing
• Build and improve CI/CD, version control, and automated testing practices for data pipelines
• Implement monitoring, alerting, logging, and operational runbooks for data pipelines to ensure reliability, timely issue detection, incident response, and continuous improvement of production data platforms
• Support AI/ML and advanced analytical workflows by provisioning trusted data, scalable infrastructure, and governed consumption patterns for data scientists, analysts, and business partners

From the employer

Optum Tech is a global leader in health care innovation. Our teams develop cutting-edge solutions that help people live healthier lives and help make the health system work better for everyone. From advanced data analytics and AI to cybersecurity, we use innovative approaches to solve some of health care's most complex challenges. Your contributions here have the potential to change lives. Ready to build the next breakthrough? Join us to start Caring. Connecting. Growing together. As a Senior Data Engineer, you will play a key role in building, optimizing, and maintaining scalable data architectures and pipelines that power critical healthcare data solutions. Functions include data architecture, engineering, modeling, integration, optimization, security, and development of cloud-based data solutions. The role designs and delivers scalable, reliable, and high-performing data platforms that support business operations, analytics, and applications. Responsibilities include ensuring data quality, governance, traceability, and compliance; automating validation and monitoring; identifying opportunities to improve business processes; and collaborating with business and technology teams. The role also provides technical guidance, establishes engineering best practices, and supports the successful delivery and operation of enterprise data solutions. You'll enjoy the flexibility to work remotely * from anywhere within the U.S. as you take on some tough challenges. For all hires in the Minneapolis or Washington, D.C. area, you will be required to work in the office a minimum of four days per week. Primary Responsibilities: • Lead the design and delivery of scalable, governed data engineering solutions, including technical direction, implementation planning, risk management, and production readiness for data products used by business, analytics, and AI/ML stakeholders • Develop and manage robust ETL/ELT pipelines for data ingestion, transformation, and integration from heterogeneous sources, including relational databases, APIs, file shares, and end-user-computing assets • Develop Python and PySpark solutions for distributed data processing, automation, integration, and reusable transformation logic • Optimize data platform performance and cloud consumption through SQL tuning, Spark job optimization, storage format selection, partitioning strategies, cluster configuration, and workload scheduling • Build and optimize workflows on Azure Cloud services and Databricks - including medallion (bronze/silver/gold) layering and orchestrated pipelines - to support enterprise data needs • Migrate business logic embedded in legacy and end-user-computing systems into documented, testable transformation logic over governed tables • Define and implement data quality, validation, reconciliation, and parity testing practices to ensure migrated pipelines are accurate, reliable, auditable, and consistent with the systems they replace • Establish data lineage, ownership, and cataloguing, and define documented rollup and allocation rules where source systems operate at differing grains • Implement security and compliance measures for handling PHI/PII and confidential business data in accordance with HIPAA or similar regulations, including de-identification and masking for controlled sharing • Build and improve CI/CD, version control, and automated testing practices for data pipelines • Implement monitoring, alerting, logging, and operational runbooks for data pipelines to ensure reliability, timely issue detection, incident response, and continuous improvement of production data platforms • Support AI/ML and advanced analytical workflows by provisioning trusted data, scalable infrastructure, and governed consumption patterns for data scientists, analysts, and business partners • Design and publish reusable data products, data services, APIs, and consumption layers that enable downstream applications, analytics platforms, and AI/ML use cases to consume trusted data consistently • Partner with product owners, business stakeholders, architects, data scientists, analysts, and engineering teams to translate requirements into scalable data solutions and communicate technical tradeoffs clearly • Design, develop, and deploy AI-enabled data solutions that address complex business challenges, support responsible AI use, and leverage AI-powered development tools to improve engineering productivity and delivery quality • Mentor data engineers and analysts through design reviews, code reviews, best-practice guidance, and support for improving team delivery quality You'll be rewarded and recognized for your performance in an environment that will challenge you and give you clear direction on what it takes to succeed in your role as well as provide development for other roles you may be interested in. Required Qualifications: • 6+ years of overall experience in data engineering, data warehousing, distributed data processing, or enterprise data platform delivery • 6+ years of experience with SQL, data modeling, and performance tuning for large-scale systems • 3+ years of experience with ETL/ELT frameworks, pipeline orchestration, and heterogeneous source-system integration • 3+ years of experience with Python for data processing, automation, and integration • 3+ years of experience with Databricks, Spark/PySpark, and Delta Lake • 3+ years of experience with public cloud platforms such as Azure or GCP • 1+ year of experience in integrating data from heterogeneous source systems • 1+ year of experience with data quality, validation, reconciliation, observability, and production support practices for large-scale data pipelines • 1+ year of experience with Git, CI/CD platforms, automated testing frameworks, release management, and deployment practices for production data pipelines • 1+ year of experience in AI-powered development tools responsibly to improve engineering productivity and delivery quality Preferred Qualifications: • Experience with batch and near-real-time data processing patterns, including event-driven ingestion, streaming pipelines, or comparable technologies where business needs require timely data availability • Experience with Azure Data Factory or comparable orchestration tooling such as Airflow, dbt, or similar • Experience migrating legacy or end-user-computing (spreadsheet- or macro-based) business logic into governed data pipelines • Experience with data governance, lineage, and catalog tooling (Unity Catalog, Microsoft Purview, or similar) • Demonstrated familiarity with AI/ML frameworks and supporting infrastructure for data science workflows • Experience working with PHI/PII and restricted client data in compliance with HIPAA or similar regulations, including de-identification and masking • Healthcare payer, actuarial, pricing, or financial-modelling domain experience *All employees working remotely will be required to adhere to UnitedHealth Group's Telecommuter Policy. Pay is based on several factors including but not limited to local labor markets, education, work experience, certifications, etc. In addition to your salary, we offer benefits such as, a comprehensive benefits package, incentive and recognition programs, equity stock purchase and 401k contribution (all benefits are subject to eligibility requirements). No matter where or when you begin a career with us, you'll find a far-reaching choice of benefits and incentives. The salary for this role will range from $91,700 - $163,700 annually based on full-time employment. We comply with all minimum wage laws as applicable. Application Deadline: This will be posted for a minimum of 2 business days or until a sufficient candidate pool has been collected. Job posting may come down early due to volume of applicants. At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission. UnitedHealth Group is an Equal Employment Opportunity employer under applicable law and qualified applicants will receive consideration for employment without regard to race, national origin, religion, age, color, sex, sexual orientation, gender identity, disability, or protected veteran status, or any other characteristic protected by local, state, or federal laws, rules, or regulations. UnitedHealth Group is a drug - free workplace. Candidates are required to pass a drug test before beginning employment.