Job description

We are looking for a skilled Senior Data Engineer to join our engineering team. In this role, you will be responsible for designing, building, and maintaining scalable data pipelines, data platforms, and data integration solutions that enable reliable analytics, machine learning, and business intelligence capabilities across the organization. Strong software engineering foundation with minimum 10+ years of experience in data engineering, backend engineering, or distributed systems. Strong hands-on experience designing and building large-scale batch and streaming data pipelines using Python and distributed systems. Expertise in ETL/ELT development, event-driven architectures, and working with high-volume datasets using technologies such as Pub/Sub, Dataflow (Apache Beam), and Spark-based processing (Dataproc or equivalent). Deep expertise in the Google Cloud Platform data ecosystem, including BigQuery, Dataflow, Pub/Sub, and Composer (Airflow). Proven ability to design, optimize, and operate scalable data platforms, with strong experience in BigQuery performance tuning, cost optimization, data modeling, partitioning, clustering, and query optimization at scale. Proficient in Python, version control, CI/CD practices, testing, and code reviews. Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical field (or equivalent practical experience). 7+ years of experience in data engineering or large-scale distributed systems. Experience designing end-to-end data platforms or data mesh architectures. Strong understanding of data governance, data lineage, and metadata management frameworks. Experience supporting ML pipelines and MLOps workflows, including feature engineering and training data generation. Experience building systems with strong reliability, observability, and SLAs (monitoring, alerting, debugging distributed pipelines). Familiarity with Generative AI / LLM-based systems, including: LLM-powered data workflows, Agentic pipelines, Embedding/vector-based retrieval systems. Experience influencing architecture decisions across teams and driving technical direction. Experience collaborating in cross-functional, fast-paced tech environments or cloud-native organizations with a focus on building reliable, maintainable, and production-grade data systems.

Requirements

Must-haves

Strong software engineering foundation with minimum 10+ years of experience in data engineering, backend engineering, or distributed systems.
Strong hands-on experience designing and building large-scale batch and streaming data pipelines using Python and distributed systems.
Expertise in ETL/ELT development, event-driven architectures, and working with high-volume datasets using technologies such as Pub/Sub, Dataflow (Apache Beam), and Spark-based processing (Dataproc or equivalent).
Proficient in Python, version control, CI/CD practices, testing, and code reviews.
Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical field (or equivalent practical experience).
7+ years of experience in data engineering or large-scale distributed systems.
Experience designing end-to-end data platforms or data mesh architectures.
Strong understanding of data governance, data lineage, and metadata management frameworks.
Experience supporting ML pipelines and MLOps workflows, including feature engineering and training data generation.
Experience building systems with strong reliability, observability, and SLAs (monitoring, alerting, debugging distributed pipelines).
Familiarity with Generative AI / LLM-based systems, including: LLM-powered data workflows, Agentic pipelines, Embedding/vector-based retrieval systems.
Experience influencing architecture decisions across teams and driving technical direction.
Experience collaborating in cross-functional, fast-paced tech environments or cloud-native organizations with a focus on building reliable, maintainable, and production-grade data systems.

Nice-to-haves

None flagged.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

machine learningpythonci/cdmlaillm

Platform: apple
Effort: medium