Job description

Create robust, scalable architectures for systems that handle data orchestration for AI features Design, build, and maintain scalable AI data platforms, services, and APIs that support and enable Agentic AI workflow development & automation. Develop data ingestion, transformation, and publishing pipelines for structured, unstructured, and multimodal data. Design and implement Retrieval-Augmented Generation (RAG) pipelines, embedding workflows, vector database integrations, and metadata services for enterprise AI applications. Be able to quickly build an idea so you and the team can work with hands-on products. Then iterate on the best of those prototypes. Build and integrate tools that help make complex AI systems observable, understandable and debuggable Strong understanding of distributed systems, parallel computing, and performance optimization Collaborate with AI/ML engineers, software engineers, product teams, and domain experts to define AI data requirements and deliver production-ready data solutions. Optimize platform scalability, reliability, performance, security, and cost across cloud-native environments and Agentic systems. Ability to clearly communicate complex technical problems and collaborate with partners to develop solutions Bachelor's or Master's degree in Computer Science, Software Engineering, Data Engineering, or a related field. 8+ Experience designing and building scalable data platforms and distributed systems. Strong programming skills in Python and SQL, with proficiency in Java or Scala preferred. Experience with Airflow, Kubeflow, or MLflow to build and orchestrate scalable AI data pipelines. Experience building scalable batch and streaming data pipelines using Spark (PySpark), Kafka, Airflow, and Ray, with proficiency in Pandas and modern data lake/lakehouse architectures (e.g., Iceberg, Delta Lake). Experience using modern development tools, including AI-assisted coding tools, while applying sound engineering judgment to review, validate, and improve generated code. Knowledge of RAG architectures, embedding generation, vector databases, and AI data preparation for LLMs and agentic AI. Experience with cloud platforms (AWS, Azure, or GCP), Kubernetes, Docker, CI/CD, and Infrastructure as Code. Demonstrated ability to understand complex user workflows, translate them into practical technical solutions, and collaborate across teams to deliver measurable outcomes. Excellent communication and collaboration skills, with the ability to translate technical concepts into clear, business focused insights. Experience with or a strong understanding of Generative AI, LLMs, Agentic Systems, or RAG (Retrieval-Augmented Generation) workflows. Experience integrating LLMs into existing systems Experience with API design, both for other engineers to use, but also for AI systems. Experience working with manufacturing, operational, IoT, or industrial data platforms. Demonstrated ability to lead technical initiatives and mentor engineers.

Requirements

Must-haves

Create robust, scalable architectures for systems that handle data orchestration for AI features
Design, build, and maintain scalable AI data platforms, services, and APIs that support and enable Agentic AI workflow development & automation.
Develop data ingestion, transformation, and publishing pipelines for structured, unstructured, and multimodal data.
Design and implement Retrieval-Augmented Generation (RAG) pipelines, embedding workflows, vector database integrations, and metadata services for enterprise AI applications.
Be able to quickly build an idea so you and the team can work with hands-on products. Then iterate on the best of those prototypes.
Build and integrate tools that help make complex AI systems observable, understandable and debuggable
Strong understanding of distributed systems, parallel computing, and performance optimization
Collaborate with AI/ML engineers, software engineers, product teams, and domain experts to define AI data requirements and deliver production-ready data solutions.
Optimize platform scalability, reliability, performance, security, and cost across cloud-native environments and Agentic systems.
Ability to clearly communicate complex technical problems and collaborate with partners to develop solutions
Bachelor's or Master's degree in Computer Science, Software Engineering, Data Engineering, or a related field.
8+ Experience designing and building scalable data platforms and distributed systems.
Experience with Airflow, Kubeflow, or MLflow to build and orchestrate scalable AI data pipelines.
Experience building scalable batch and streaming data pipelines using Spark (PySpark), Kafka, Airflow, and Ray, with proficiency in Pandas and modern data lake/lakehouse architectures (e.g., Iceberg, Delta Lake).
Experience using modern development tools, including AI-assisted coding tools, while applying sound engineering judgment to review, validate, and improve generated code.
Knowledge of RAG architectures, embedding generation, vector databases, and AI data preparation for LLMs and agentic AI.
Experience with cloud platforms (AWS, Azure, or GCP), Kubernetes, Docker, CI/CD, and Infrastructure as Code.
Demonstrated ability to understand complex user workflows, translate them into practical technical solutions, and collaborate across teams to deliver measurable outcomes.
Excellent communication and collaboration skills, with the ability to translate technical concepts into clear, business focused insights.
Experience with or a strong understanding of Generative AI, LLMs, Agentic Systems, or RAG (Retrieval-Augmented Generation) workflows.
Experience integrating LLMs into existing systems
Experience with API design, both for other engineers to use, but also for AI systems.
Experience working with manufacturing, operational, IoT, or industrial data platforms.

Nice-to-haves

Strong programming skills in Python and SQL, with proficiency in Java or Scala preferred.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

aimlpythonsqljavaawsazuregcpkubernetesdockerci/cdapi

Platform: apple
Effort: medium