Job description

Experience: 12+ Years About the role We are looking for a hands-on technical lead who can hold design authority on an enterprise AWS data lakehouse and build on it personally. The person in this seat writes production code, authors the architecture that the team builds to, owns the governance and security model, and prototypes new AI capability directly. You will be the senior technical voice in front of a large enterprise client, working alongside their data strategy, governance, and AI leadership as well as AWS specialists. You will also guide a distributed engineering team, review their work for design intent rather than only correctness, and lift their standard. The role moves up and down the stack by design. It may involve authoring a solution design document for client sign-off, debugging a cross-account access path, building a retrieval pipeline in a sandbox, and presenting a cost and architecture recommendation to client leadership. What you will do Data platform engineering • Design and build data pipelines across a medallion lakehouse architecture, from source ingestion through curated, governed data products. • Build on the AWS data stack directly: S3 and S3 Tables, Glue, Athena, Lambda, Step Functions, DMS-based change data capture, and Airflow or MWAA for orchestration. • Work with Apache Iceberg at a level that includes table format versions, catalog models, and the query-engine compatibility consequences of each. • Deliver all infrastructure as code using AWS CDK. Manual console changes are treated as defects, not shortcuts. • Debug across account and service boundaries, including IAM and catalog permission paths, cross-account access, identity federation, and orchestration failures. Applied AI and GenAI delivery • Build agentic and conversational data access on Amazon Bedrock, including agent runtimes, tool design, and natural language to query translation over governed data. • Design and build retrieval systems: embedding models, vector search, chunking strategy, ontology design, and document intelligence over unstructured enterprise content. • Engineer provenance and trust into AI output so that client-facing results are defensible about what was extracted deterministically and what was inferred. • Model and measure real end-to-end inference and infrastructure cost, and use those numbers in architecture and commercial recommendations. Governance and security • Own the data access control model, including role-based and attribute-based grant strategies, row and column-level filtering, and enforcement tiers. • Design and implement identity federation across enterprise identity providers, AWS IAM Identity Center, and the data catalog layer, including trusted identity propagation. • Assess third-party tooling for governance compatibility before procurement, including whether per-user enforcement is preserved end to end. • Surface security exposure proactively and record decisions, including where the answer is to accept and schedule remediation rather than block. Technical leadership • Author and maintain solution design documents, architectural decision records, coding standards, and test strategy. These are client-signed artifacts, not internal notes. • Run option evaluations to a decision: verify vendor and service claims independently, recommend with reasoning, and document the conditions under which the decision should be revisited. • Detect and escalate drift between documented architecture and what is actually being built. • Review engineering output for design intent, security posture, and infrastructure discipline. Mentor engineers toward a higher standard rather than correcting output after the fact. • Break work down into clear, single-sprint stories with explicit acceptance criteria, and support sprint planning and estimation. Client engagement • Act as the senior technical counterpart to client architecture and data leadership. • Run design workshops and working sessions, and capture decisions, owners, and actions. • Present architecture, cost, and options to client leadership clearly and briefly. • Support scope definition and effort estimation for future phases of work. Required skills and experience • 10+ years in technology delivery, including 4+ years as a technical lead, architect, or senior engineer on data platform or data engineering programs. • Deep, hands-on AWS experience: S3, S3 Tables, Glue, Athena, Lambda, Step Functions, DMS, Lake Formation, IAM, API Gateway, CloudFront, Secrets Manager, CloudWatch, and ECR. You should be able to read CloudTrail and resolve an access-denied path unaided. • Apache Iceberg: practical working knowledge of table formats, catalogs, and engine compatibility. • AWS CDK (Python): you build and maintain infrastructure as code as a matter of course. • Python at production standard: Lambda handlers, PySpark ETL, CDK stacks, and test suites that you write and review, not merely read. • Orchestration with Airflow or MWAA. • Enterprise access control at scale: RBAC and ABAC models, fine-grained filtering, identity federation, and the operational reality of managing grants across many principals and resources. • Experience leading distributed engineering teams across time zones, with Git and disciplined commit and review practice. • This role produces a significant volume of design documentation and client-facing material, and the quality of that writing matters. Preferred • GenAI delivery experience: Amazon Bedrock, agent runtimes, embedding models, vector search, and RAG system design. • Document intelligence: OCR, layout-aware extraction, chunking strategy, and ontology design over unstructured content. • Identity federation with Microsoft Entra ID and AWS IAM Identity Center, including trusted identity propagation. • SageMaker Unified Studio, Amazon DataZone, or comparable data catalog and subscription platforms. • Semantic layer and BI experience with Power BI, Cognos, or equivalent, including correct treatment of ratio measures across aggregation levels. • React and FastAPI, sufficient to build and maintain internal tooling. • Data cataloguing and data privacy platforms such as Atlan or BigID. • Consulting or client-services delivery within a large enterprise account. How we work • Establish ground truth before building. Audit the live environment first and separate what was measured from what was assumed. • Verify against real state, not a green pipeline. A successful deployment is not evidence that the thing works. • Raise uncertainty early. Stopping to ask is always preferred over guessing and continuing. • Infrastructure as code, without exception. Manual changes are permitted only as approved, temporary steps to validate a fix before it is codified. • Leave no trace in shared environments, and be able to prove it.

Requirements

Must-haves

Experience: 12+ Years
Data platform engineering
Design and build data pipelines across a medallion lakehouse architecture, from source ingestion through curated, governed data products.
Build on the AWS data stack directly: S3 and S3 Tables, Glue, Athena, Lambda, Step Functions, DMS-based change data capture, and Airflow or MWAA for orchestration.
Work with Apache Iceberg at a level that includes table format versions, catalog models, and the query-engine compatibility consequences of each.
Deliver all infrastructure as code using AWS CDK. Manual console changes are treated as defects, not shortcuts.
Debug across account and service boundaries, including IAM and catalog permission paths, cross-account access, identity federation, and orchestration failures.
Applied AI and GenAI delivery
Build agentic and conversational data access on Amazon Bedrock, including agent runtimes, tool design, and natural language to query translation over governed data.
Design and build retrieval systems: embedding models, vector search, chunking strategy, ontology design, and document intelligence over unstructured enterprise content.
Engineer provenance and trust into AI output so that client-facing results are defensible about what was extracted deterministically and what was inferred.
Model and measure real end-to-end inference and infrastructure cost, and use those numbers in architecture and commercial recommendations.

Nice-to-haves

GenAI delivery experience: Amazon Bedrock, agent runtimes, embedding models, vector search, and RAG system design.
Document intelligence: OCR, layout-aware extraction, chunking strategy, and ontology design over unstructured content.
Identity federation with Microsoft Entra ID and AWS IAM Identity Center, including trusted identity propagation.
SageMaker Unified Studio, Amazon DataZone, or comparable data catalog and subscription platforms.
Semantic layer and BI experience with Power BI, Cognos, or equivalent, including correct treatment of ratio measures across aggregation levels.
React and FastAPI, sufficient to build and maintain internal tooling.
Data cataloguing and data privacy platforms such as Atlan or BigID.
Consulting or client-services delivery within a large enterprise account.
Establish ground truth before building. Audit the live environment first and separate what was measured from what was assumed.
Verify against real state, not a green pipeline. A successful deployment is not evidence that the thing works.
Raise uncertainty early. Stopping to ask is always preferred over guessing and continuing.
Infrastructure as code, without exception. Manual changes are permitted only as approved, temporary steps to validate a fix before it is codified.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

awsaiapipythonreact

Platform: careers-browser
Effort: medium