Senior Data Engineer at EXL

Job description

EXL is hiring a Senior Data Engineer to join a strategic AI / ML platform engagement with a leading specialty retailer. This is a hands-on build role embedded with the client's platform engineering team. The role requires shipping production-grade data pipelines that feed real-time customer event data into machine learning workflows. The right person is comfortable owning the full lifecycle of pipeline design, build, and deployment: from streaming ingestion through event store design to model-ready feature delivery. This is a high-visibility role with growth potential into a larger book of work as the engagement expands. Salary Range: $93,900 - $154,200 annual base The posted range is the hiring range for this role — a subset of the broader range available to employees over time — and reflects base salary across our national hiring scale. Final offers are based on several factors, including the candidate's skills and experience, internal pay equity, work location, market conditions for the role, and the specific scope and responsibilities of the position. The top of the range is reserved for candidates who notably exceed the requirements; the lower end applies to those with less experience or fewer preferred qualifications. For positions based in higher-cost zones (e.g., California, New York, New Jersey), actual compensation may exceed the posted range; your recruiter will share specifics during the process. What You'll Do • Design and operate event-driven data pipelines using Kafka consumers and Flink jobs to process high-volume customer events (clicks, purchases, returns) in near-real time. • Build and optimize large-scale data transformations on Google Cloud Platform — BigQuery SQL, query performance tuning, and partitioning strategy at scale. • Develop Python data engineering workloads using Polars or Pandas at scale, with rigorous attention to Parquet partitioning, join performance on large datasets, and memory efficiency. • Build, deploy, and maintain ML pipeline components on Kubeflow Pipelines (KFP) and Vertex AI; package and deploy services with Docker. • Design event store architecture: partitioning by customer, time-ordered event assembly across heterogeneous sources, and schema management for mixed event types. • Partner with ML engineers, platform engineers, and data scientists to deliver clean, performant, model-ready data products. • Document architecture decisions and contribute to engineering standards across the platform team. Required Skills & Experience • 6–12 years of experience in data engineering, platform engineering, or a closely related discipline. • Streaming: Production experience with Kafka consumers and Flink stream processing — building, deploying, and operating streaming jobs at meaningful scale. • GCP Data Stack: Strong SQL on BigQuery (or an equivalent cloud warehouse), with demonstrated query optimization, cost management, and partitioning chops. • Python Data Engineering: Hands-on with Polars or Pandas at scale; deep working knowledge of Parquet partitioning and performance on large joins. • ML Pipelines: Hands-on experience building and deploying components on Kubeflow Pipelines (KFP) and/or Vertex AI Pipelines; working proficiency with Docker. • Event Store Design: Demonstrated experience designing event stores — partitioning by customer, time-ordered event assembly across sources, schema strategy for mixed event types (clicks, purchases, returns). • Communication: Strong written and verbal communication; comfortable being the senior IC voice in design conversations with client stakeholders. Nice to Have • Domain experience in Retail or E-commerce — customer journey data, transaction analytics, returns and exchanges modeling. • Exposure to schema registry tooling (e.g., Confluent), Iceberg, or Delta Lake. • Experience working in client-facing or consulting engagements. • Google Cloud certifications (Professional Data Engineer or equivalent). Work Arrangement & Eligibility • This role requires 3–4 days per week onsite in Seattle, WA. Fully remote and out-of-state candidates will not be considered. • EXL is open to sponsoring H1B transfers for qualified candidates. EXL is hiring a Senior Data Engineer to join a strategic AI / ML platform engagement with a leading specialty retailer. This is a hands-on build role embedded with the client's platform engineering team.

Requirements

Must-haves

EXL is hiring a Senior Data Engineer to join a strategic AI / ML platform engagement with a leading specialty retailer. This is a hands-on build role embedded with the client's platform engineering team.
This is a high-visibility role with growth potential into a larger book of work as the engagement expands.
Design and operate event-driven data pipelines using Kafka consumers and Flink jobs to process high-volume customer events (clicks, purchases, returns) in near-real time.
Build and optimize large-scale data transformations on Google Cloud Platform — BigQuery SQL, query performance tuning, and partitioning strategy at scale.
Develop Python data engineering workloads using Polars or Pandas at scale, with rigorous attention to Parquet partitioning, join performance on large datasets, and memory efficiency.
Build, deploy, and maintain ML pipeline components on Kubeflow Pipelines (KFP) and Vertex AI; package and deploy services with Docker.
Design event store architecture: partitioning by customer, time-ordered event assembly across heterogeneous sources, and schema management for mixed event types.
Partner with ML engineers, platform engineers, and data scientists to deliver clean, performant, model-ready data products.
Document architecture decisions and contribute to engineering standards across the platform team.
Required Skills & Experience
6–12 years of experience in data engineering, platform engineering, or a closely related discipline.
Streaming: Production experience with Kafka consumers and Flink stream processing — building, deploying, and operating streaming jobs at meaningful scale.
GCP Data Stack: Strong SQL on BigQuery (or an equivalent cloud warehouse), with demonstrated query optimization, cost management, and partitioning chops.
Python Data Engineering: Hands-on with Polars or Pandas at scale; deep working knowledge of Parquet partitioning and performance on large joins.
ML Pipelines: Hands-on experience building and deploying components on Kubeflow Pipelines (KFP) and/or Vertex AI Pipelines; working proficiency with Docker.
Event Store Design: Demonstrated experience designing event stores — partitioning by customer, time-ordered event assembly across sources, schema strategy for mixed event types (clicks, purchases, returns).

Nice-to-haves

Domain experience in Retail or E-commerce — customer journey data, transaction analytics, returns and exchanges modeling.
Exposure to schema registry tooling (e.g., Confluent), Iceberg, or Delta Lake.
Experience working in client-facing or consulting engagements.
Google Cloud certifications (Professional Data Engineer or equivalent).
Work Arrangement & Eligibility
This role requires 3–4 days per week onsite in Seattle, WA. Fully remote and out-of-state candidates will not be considered.
EXL is open to sponsoring H1B transfers for qualified candidates.
EXL is hiring a Senior Data Engineer to join a strategic AI / ML platform engagement with a leading specialty retailer. This is a hands-on build role embedded with the client's platform engineering team.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

aimlmachine learningsqlpythondockergcp

Platform: oracle-cloud
Effort: medium