Job description

We're looking for a Senior Data Engineer to design and implement AI features end to end — from data pipelines through GenAI and agentic workflows to production deployment. In this role, you'll build and optimize scalable, secure data pipelines across polyglot client enterprise environments, develop Generative AI, agentic, ML, and BI systems using modern frameworks, and implement MLOps/LLMOps practices across AWS, Azure, and GCP. You'll own technical workstreams autonomously, mentor junior engineers, and bring sound judgment on model behavior, evaluation, reliability, and guardrails to everything you build. Responsibilities: • Help design and implement AI features end to end — from data pipelines through GenAI/agentic workflows to production deployment — applying sound judgment on model behavior, evaluation, reliability, and guardrails. • Build and optimize scalable, secure data pipelines and ETL/ELT workflows in Python and SQL across polyglot client enterprise environments. • Develop Generative AI, agentic, ML, and BI systems using RAG, embeddings, vector databases, and modern frameworks (Spark, LangChain, Databricks, Airflow, and similar). • Implement MLOps/LLMOps practices — CI/CD for data workflows, automated agent evaluation, and infrastructure as code across AWS, Azure, and GCP. • Enforce data security and governance, including PII/PHI handling, authentication, and role-based access control. • Own technical workstreams autonomously, mentor junior engineers, and champion software engineering best practices across the team. • Active Secret or Top Secret clearance required. • 5+ years of relevant professional experience in a data engineering role, including leading technical workstreams, mentoring junior engineers, and driving adoption of software engineering best practices within a team. • Expert-level proficiency in Python and SQL, with the ability to work in polyglot environments (Scala, Java) as required by client enterprise systems. • Strong experience building Agentic AI, Generative AI, Machine Learning, and Business Intelligence systems — including prompt design, retrieval-augmented generation (RAG), embeddings, vector databases, context construction, and output handling in production workflows using modern frameworks (Spark, LangChain, Databricks, Dask, Airflow, Dagster, Kedro, etc.). • Ability to lead the implementation of AI features end to end, with sound judgment around model behavior, evaluation, reliability, guardrails, and the trade-offs between quality, latency, and cost. • Experience implementing robust data security and governance controls, including managing PII/PHI, authentication, and role-based access control (RBAC). • Deep knowledge of MLOps/LLMOps, including CI/CD for data workflows, automated agent evaluation (LangSmith, Opik, Langfuse), and infrastructure as code (Terraform) across cloud providers (AWS, Azure, GCP). • Exceptional time management and the ability to own technical workstreams autonomously. We Value: • Drive: Passion and energy to implement quality technical solutions. Self-motivation and intellectual curiosity • Commitment to Quality: Passion to conceive and produce world-class solutions that drive real-world value for the customer • Customer Focus: Consultative approach to solving problems for customers. Expectations management. • Communication: Superior communication skills. Ability to clearly articulate problems, solutions, risks, rewards etc. (written and verbal) • Technical Skills: Love for technology. You have to be inherently passionate about technology. • Business Acumen: Technology ultimately is used to enable the business. We look for people who understand how the businesses can be enabled through their technical solutions All your information will be kept confidential according to EEO guidelines.

Requirements

Must-haves

Help design and implement AI features end to end — from data pipelines through GenAI/agentic workflows to production deployment — applying sound judgment on model behavior, evaluation, reliability, and guardrails.
Build and optimize scalable, secure data pipelines and ETL/ELT workflows in Python and SQL across polyglot client enterprise environments.
Develop Generative AI, agentic, ML, and BI systems using RAG, embeddings, vector databases, and modern frameworks (Spark, LangChain, Databricks, Airflow, and similar).
Implement MLOps/LLMOps practices — CI/CD for data workflows, automated agent evaluation, and infrastructure as code across AWS, Azure, and GCP.
Enforce data security and governance, including PII/PHI handling, authentication, and role-based access control.
Own technical workstreams autonomously, mentor junior engineers, and champion software engineering best practices across the team.
Active Secret or Top Secret clearance required.
5+ years of relevant professional experience in a data engineering role, including leading technical workstreams, mentoring junior engineers, and driving adoption of software engineering best practices within a team.
Expert-level proficiency in Python and SQL, with the ability to work in polyglot environments (Scala, Java) as required by client enterprise systems.
Ability to lead the implementation of AI features end to end, with sound judgment around model behavior, evaluation, reliability, guardrails, and the trade-offs between quality, latency, and cost.
Experience implementing robust data security and governance controls, including managing PII/PHI, authentication, and role-based access control (RBAC).
Deep knowledge of MLOps/LLMOps, including CI/CD for data workflows, automated agent evaluation (LangSmith, Opik, Langfuse), and infrastructure as code (Terraform) across cloud providers (AWS, Azure, GCP).
Exceptional time management and the ability to own technical workstreams autonomously.
Drive: Passion and energy to implement quality technical solutions. Self-motivation and intellectual curiosity
Commitment to Quality: Passion to conceive and produce world-class solutions that drive real-world value for the customer
Customer Focus: Consultative approach to solving problems for customers. Expectations management.
Communication: Superior communication skills. Ability to clearly articulate problems, solutions, risks, rewards etc. (written and verbal)
Technical Skills: Love for technology. You have to be inherently passionate about technology.
Business Acumen: Technology ultimately is used to enable the business. We look for people who understand how the businesses can be enabled through their technical solutions
All your information will be kept confidential according to EEO guidelines.

Nice-to-haves

None flagged.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

aimlawsazuregcppythonsqlci/cdjavamachine learning

Platform: smartrecruiters
Effort: medium