Job description

• Develop and automate large scale, high-performance, scalable platform (batch and/or streaming) to drive faster analytics • Ability to design large-scale, complex applications and frameworks with excellent run-time characteristics such as low-latency, fault-tolerance and availability • Experience in building and maintaining custom frameworks to support engineering/analytics needs • Knowledge of continuous integration, testing methodologies, TDD and agile development methodologies. • Partner with analytic consumers and data scientists to build and improve new/existing constructs and solve data engineering problems @ scale. • Experience in building data pipelines in Spark, Trino, lakehouse or similar distributed platforms & Snowflake. • Deploy inclusive data quality checks to ensure high quality of data. • Evangelize high quality software engineering practices towards building data infrastructure and pipelines at scale. • Structured thinking with ability to easily break down ambiguous problems and propose impactful solutions. • Applying Generative AI and Retrieval Augmented Generation (RAG) techniques to enhance data analytics capabilities • Communication Strong documentation and technical writing skills. • Attention to detail and effective verbal/written communication skills. 5+ years of relevant Data Engineering experience Undergraduate degree in Computer Science, MIS, Engineering, Mathematics or other quantitative discipline required. 5+ years of experience in data engineering and ETL pipeline development 5+ years of experience in Big Data Technologies (Spark,Lakehouse,Trino) Experience on Kubernetes, Docker preferred. Familiarity with Retrieval Augmented Generation (RAG) techniques working in conjunction with LLMs Experience with creating and consuming Model Context Protocol (MCP) services Experience with Snowflake

Requirements

Must-haves

Develop and automate large scale, high-performance, scalable platform (batch and/or streaming) to drive faster analytics
Ability to design large-scale, complex applications and frameworks with excellent run-time characteristics such as low-latency, fault-tolerance and availability
Experience in building and maintaining custom frameworks to support engineering/analytics needs
Knowledge of continuous integration, testing methodologies, TDD and agile development methodologies.
Partner with analytic consumers and data scientists to build and improve new/existing constructs and solve data engineering problems @ scale.
Experience in building data pipelines in Spark, Trino, lakehouse or similar distributed platforms & Snowflake.
Deploy inclusive data quality checks to ensure high quality of data.
Evangelize high quality software engineering practices towards building data infrastructure and pipelines at scale.
Structured thinking with ability to easily break down ambiguous problems and propose impactful solutions.
Applying Generative AI and Retrieval Augmented Generation (RAG) techniques to enhance data analytics capabilities
Communication Strong documentation and technical writing skills.
Attention to detail and effective verbal/written communication skills.
5+ years of relevant Data Engineering experience
Undergraduate degree in Computer Science, MIS, Engineering, Mathematics or other quantitative discipline required.
5+ years of experience in data engineering and ETL pipeline development
5+ years of experience in Big Data Technologies (Spark,Lakehouse,Trino)
Familiarity with Retrieval Augmented Generation (RAG) techniques working in conjunction with LLMs
Experience with creating and consuming Model Context Protocol (MCP) services
Experience with Snowflake

Nice-to-haves

Experience on Kubernetes, Docker preferred.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

agileaikubernetesdocker

Platform: apple
Effort: medium