Job description

Job Description: As a Data Engineer III at JPMorganChase within the Corporate Sector you serve as a seasoned member of an agile team to design and deliver trusted data collection, storage, access, and analytics solutions in a secure, stable, and scalable way. As a Data Engineer, you will be responsible for designing, building, and maintaining robust data pipelines and architectures that support the entire data engineering lifecycle. You will work closely with cross-functional teams to deliver scalable solutions that drive business insights and enable advanced analytics, including AI initiatives. Key Responsibilities: • Design, develop, and optimize end-to-end data pipelines using Databricks, Apache Spark, and AWS cloud services. • Implement and manage workflow orchestration with Airflow. • Integrate and process streaming data using Kafka. • Uses enterprise-authorized AI capabilities within the work environment to accelerate data pipeline/design analysis and documentation, validating outputs and handling data according to sensitivity and security requirements. • Collaborate with data scientists, analysts, and business stakeholders to understand requirements and deliver high-quality data solutions. • Ensure data quality, integrity, and security across all stages of the data lifecycle. • Develop and maintain ETL processes, data models, and data warehouses. • Write efficient, maintainable code in Python and SQL for data processing and analysis. • Applies reuse-first, AI-assisted practices to strengthen SDLC-quality routines for data pipelines (e.g., test generation and control validation), ensuring traceability/auditability and alignment to resiliency and security expectations. • Support AI and machine learning initiatives by preparing and engineering data for model development and deployment. • Monitor, troubleshoot, and optimize data workflows for performance and reliability. • Stay current with industry best practices and emerging technologies in data engineering and AI. Required qualifications, capabilities, and skills • Formal training or certification on data engineering concepts and 3+ years applied experience • Proven experience in the full lifecycle of data engineering, from ingestion to production. • Demonstrated experience using enterprise-authorized AI capabilities within the work environment to support data engineering workflows with strong validation habits and awareness of data sensitivity. • Ability to review and validate AI-assisted outputs (e.g., query suggestions, test ideas, or model change summaries) before use, escalating when uncertain and following data handling requirements. • Strong hands-on expertise with Databricks, Apache Spark, AWS (S3, Redshift, Lambda, etc.), Airflow, and Kafka. • Proficient in Python programming and SQL for data manipulation and analysis. • Experience with data modeling, ETL development, and data warehousing. • Familiarity with AI/ML workflows and supporting data pipelines for machine learning. • Certifications in relevant technologies (AWS, Databricks, Spark, etc.) are highly desirable. • Excellent problem-solving skills and attention to detail. • Strong communication and teamwork abilities. Preferred qualifications, capabilities, and skills • Experience with containerization (Docker, Kubernetes). • Knowledge of data governance and compliance standards. • Exposure to big data tools and frameworks. • Ability to work in a fast-paced, agile environment. Be part of a dynamic team where your distinctive skills will contribute to a winning culture and team

Requirements

Must-haves

Design, develop, and optimize end-to-end data pipelines using Databricks, Apache Spark, and AWS cloud services.
Implement and manage workflow orchestration with Airflow.
Integrate and process streaming data using Kafka.
Collaborate with data scientists, analysts, and business stakeholders to understand requirements and deliver high-quality data solutions.
Ensure data quality, integrity, and security across all stages of the data lifecycle.
Develop and maintain ETL processes, data models, and data warehouses.
Write efficient, maintainable code in Python and SQL for data processing and analysis.
Support AI and machine learning initiatives by preparing and engineering data for model development and deployment.
Monitor, troubleshoot, and optimize data workflows for performance and reliability.
Stay current with industry best practices and emerging technologies in data engineering and AI.
Required qualifications, capabilities, and skills
Formal training or certification on data engineering concepts and 3+ years applied experience
Proven experience in the full lifecycle of data engineering, from ingestion to production.
Demonstrated experience using enterprise-authorized AI capabilities within the work environment to support data engineering workflows with strong validation habits and awareness of data sensitivity.
Ability to review and validate AI-assisted outputs (e.g., query suggestions, test ideas, or model change summaries) before use, escalating when uncertain and following data handling requirements.
Strong hands-on expertise with Databricks, Apache Spark, AWS (S3, Redshift, Lambda, etc.), Airflow, and Kafka.
Proficient in Python programming and SQL for data manipulation and analysis.
Experience with data modeling, ETL development, and data warehousing.
Familiarity with AI/ML workflows and supporting data pipelines for machine learning.
Certifications in relevant technologies (AWS, Databricks, Spark, etc.) are highly desirable.
Excellent problem-solving skills and attention to detail.
Strong communication and teamwork abilities.
Experience with containerization (Docker, Kubernetes).

Nice-to-haves

Preferred qualifications, capabilities, and skills

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

agileaiawspythonsqlmachine learningmldockerkubernetes

Platform: oracle-cloud
Effort: medium