Job description

We are seeking an experienced Data Engineer with deep expertise in ETL/ELT, data architecture, and applied ML pipelines to design, build, and operate this infrastructure. As a key member of the team, you will be responsible for creating the massively scalable pipelines that turn raw data into a trusted foundation, driving critical decision-making and operations across the entire system. Masters Degree 10+ years of experience in data engineering, including building and maintaining large-scale ETL/ELT data pipelines Proficiency in data modeling, especially dimensional modeling, and designing schemas optimized for analytics and reporting Experience with leveraging databases including SQL/NoSQL Databases (including Postgres / Cassandra / Redis) Strong experience with distributed data processing frameworks including Apache Spark Strong experience with Parallel processing frameworks: BigTable/Hadoop Strong software engineering fundamentals and proven experience with Scala, Java Hands-on experience with Apache Kafka, Iceberg, and Flink. Experience with workflow orchestration tools including Apache Airflow and Beam Experience with AWS: e.g., S3, EMR, Lambda, Glue, Redshift, BigQuery, Kinesis, or similar services Experience with Analytics frameworks including Trino (Presto, BigQuery, Snowflake) Hands-on experience with big data lake architectures Experience with containerization and orchestration (Docker, Kubernetes/EKS) and CI/CD tooling including Jenkins Experience in Python and PySpark Familiarity with graph databases such as TigerGraph Experience building pipelines that process multimodal data (structured and image) and integrate ML model inference - including LLMs and embedding models - for data enrichment and transformation Hands-on experience deploying, serving, and optimizing LLMs or ML models directly in the production, inference runtimes/compilers (ONNX Runtime, TensorRT/TensorRT-LLM), and serving frameworks (Triton, vLLM, TorchServe or similar). Experience tuning batching, KV-cache, and GPU utilization for low-latency, high-throughput real-time inference in a data pipeline Knowledge of data governance principles, data security best practices, and data privacy regulations Experience with data versioning tools and frameworks (e.g., DVC, Delta Lake) Excellent communication skills and a collaborative mindset Experience storing/serving embeddings (e.g., pgvector, Milvus, FAISS)

Requirements

Must-haves

Masters Degree
10+ years of experience in data engineering, including building and maintaining large-scale ETL/ELT data pipelines
Proficiency in data modeling, especially dimensional modeling, and designing schemas optimized for analytics and reporting
Experience with leveraging databases including SQL/NoSQL Databases (including Postgres / Cassandra / Redis)
Strong experience with distributed data processing frameworks including Apache Spark
Strong experience with Parallel processing frameworks: BigTable/Hadoop
Strong software engineering fundamentals and proven experience with Scala, Java
Hands-on experience with Apache Kafka, Iceberg, and Flink.
Experience with workflow orchestration tools including Apache Airflow and Beam
Experience with AWS: e.g., S3, EMR, Lambda, Glue, Redshift, BigQuery, Kinesis, or similar services
Experience with Analytics frameworks including Trino (Presto, BigQuery, Snowflake)
Hands-on experience with big data lake architectures
Experience with containerization and orchestration (Docker, Kubernetes/EKS) and CI/CD tooling including Jenkins
Experience in Python and PySpark
Familiarity with graph databases such as TigerGraph
Experience building pipelines that process multimodal data (structured and image) and integrate ML model inference - including LLMs and embedding models - for data enrichment and transformation
Experience tuning batching, KV-cache, and GPU utilization for low-latency, high-throughput real-time inference in a data pipeline
Knowledge of data governance principles, data security best practices, and data privacy regulations
Experience with data versioning tools and frameworks (e.g., DVC, Delta Lake)
Excellent communication skills and a collaborative mindset
Experience storing/serving embeddings (e.g., pgvector, Milvus, FAISS)

Nice-to-haves

None flagged.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

mlsqlpostgresjavaawsdockerkubernetesci/cdpythonllm

Platform: apple
Effort: medium