Job description

Description: This role requires working onsite 4 days per week, and a F2F interview at the client’s Jersey City location is mandatory. Only USCs/GCs are eligible POSITION OVERVIEW : Distinguished AI Engineer POSITION GENERAL DUTIES AND TASKS : Distinguished AI Engineer Enterprise AI Platform Role purpose Provide top-tier individual-contributor technical leadership for AIRP. The role solves complex AI platform engineering problems and drives reusable, secure, reliable, observable, and cost-efficient capabilities that enable teams to deliver enterprise AI and GenAI solutions at scale. The role leads through deep technical judgment, working engineering assets, and influencenot direct people management. Primary ownership Technical evolution and adoption of shared AIRP capabilities: AI gateways, model access and serving, RAG, agent/tool execution, orchestration, evaluation, observability, and LLMOps/MLOps. Engineering outcomes for critical platform capabilities: reliability, performance, scalability, security, cost efficiency, operational readiness, and developer experience. Reusable engineering paved roads, including reference implementations, infrastructure templates, CI/CD patterns, evaluation harnesses, and operational tooling. Technical leadership for complex, high-impact AI initiatives where deep design, prototype validation, performance analysis, or production troubleshooting is required. Key responsibilities Lead the design and engineering evolution of AIRP services for model access, routing, serving, RAG, agents, orchestration, evaluation, and observability. Solve difficult engineering trade-offs across capability, latency, throughput, resiliency, security, data isolation, portability, and cost. Lead critical technical spikes, prototypes, reference implementations, deep design reviews, and resolution of major platform or production issues. Establish measurable engineering expectations for availability, recovery, performance, capacity, telemetry, release safety, evaluation coverage, and inference cost. Build or sponsor reusable engineering assets: APIs, SDKs, Terraform/IaC modules, deployment patterns, CI/CD templates, dashboards, evaluation tooling, and developer guidance. Drive production excellence through observability, traceability, controlled releases, rollback, incident learning, capacity planning, and cost optimization. Engineer security and AI controls into platform capabilities, including identity and access controls, authorization-aware retrieval, secure tool execution, prompt-injection defenses, data protection, logging, and audit evidence. Partner with cybersecurity, risk, compliance, legal, audit, architecture, product, and business teams to translate enterprise requirements into usable technical controls and delivery patterns. Assess emerging AI technologies through practical technical evaluation and recommend adoption based on value, maturity, risk, operational fit, and total cost of ownership. Mentor senior engineers and drive adoption of AIRP patterns across teams that do not report directly to the role. Required candidate profile 10+ years of progressive experience in software engineering, distributed systems, cloud/platform engineering, AI/ML infrastructure, or related technical domains, with substantial experience leading complex systems across multiple teams. Demonstrated record of building, scaling, transforming, or rescuing production platforms or critical technical systems used by multiple engineering teams, products, or business domains. Deep hands-on expertise in several of the following: LLM platforms, model serving, inference optimization, AI gateways, model routing, RAG, embeddings/vector search, agent frameworks, tool execution, AI orchestration, LLMOps/MLOps, evaluation systems, AI observability, or AI security controls.\ Strong engineering foundation in distributed systems, API/platform design, cloud-native architecture, containers/Kubernetes, networking, IAM, secrets management, high ava

Requirements

Must-haves

This role requires working onsite 4 days per week, and a F2F interview at the client’s Jersey City location is mandatory.
Only USCs/GCs are eligible
POSITION OVERVIEW : Distinguished AI Engineer POSITION GENERAL DUTIES AND TASKS : Distinguished AI Engineer Enterprise AI Platform
Role purpose
Primary ownership
Technical evolution and adoption of shared AIRP capabilities: AI gateways, model access and serving, RAG, agent/tool execution, orchestration, evaluation, observability, and LLMOps/MLOps.
Engineering outcomes for critical platform capabilities: reliability, performance, scalability, security, cost efficiency, operational readiness, and developer experience.
Reusable engineering paved roads, including reference implementations, infrastructure templates, CI/CD patterns, evaluation harnesses, and operational tooling.
Technical leadership for complex, high-impact AI initiatives where deep design, prototype validation, performance analysis, or production troubleshooting is required.
Lead the design and engineering evolution of AIRP services for model access, routing, serving, RAG, agents, orchestration, evaluation, and observability.
Solve difficult engineering trade-offs across capability, latency, throughput, resiliency, security, data isolation, portability, and cost.
Lead critical technical spikes, prototypes, reference implementations, deep design reviews, and resolution of major platform or production issues.
Establish measurable engineering expectations for availability, recovery, performance, capacity, telemetry, release safety, evaluation coverage, and inference cost.
Build or sponsor reusable engineering assets: APIs, SDKs, Terraform/IaC modules, deployment patterns, CI/CD templates, dashboards, evaluation tooling, and developer guidance.
Drive production excellence through observability, traceability, controlled releases, rollback, incident learning, capacity planning, and cost optimization.
Partner with cybersecurity, risk, compliance, legal, audit, architecture, product, and business teams to translate enterprise requirements into usable technical controls and delivery patterns.
Assess emerging AI technologies through practical technical evaluation and recommend adoption based on value, maturity, risk, operational fit, and total cost of ownership.
Mentor senior engineers and drive adoption of AIRP patterns across teams that do not report directly to the role.
Required candidate profile
Demonstrated record of building, scaling, transforming, or rescuing production platforms or critical technical systems used by multiple engineering teams, products, or business domains.
Strong engineering foundation in distributed systems, API/platform design, cloud-native architecture, containers/Kubernetes, networking, IAM, secrets management, high ava

Nice-to-haves

None flagged.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

aici/cdmlllmapikubernetes

Platform: jobdiva
Effort: medium