Job description

Our client, an IT Services and Consulting company, is looking for a Senior Data Engineer – AWS, Databricks & Palantir Foundry for their Remote location. Responsibilities: • Architect role in a global enterprise focusing on design and implementation of scalable data platforms using AWS, Databricks and Palantir Foundry in a hybrid work model for day shift with no travel requirement, ensuring secure, reliable and cost efficient solutions that align with business outcomes and social impact objectives. • Design robust data architectures on AWS that support analytics, reporting and machine learning workloads while ensuring scalability, performance and security compliance • Develop end to end data processing solutions using Databricks that optimize data ingestion, transformation and aggregation for diverse business domains • Coordinate integration of Palantir Foundry platforms with existing enterprise systems to enable unified data views and reusable analytical assets • Implement data modelling practices that standardize reference data, master data and domain specific schemas to support consistent decision making across teams • Optimize cloud resource usage on AWS by refining storage patterns, compute configurations and scheduling strategies to balance cost and performance • Create reusable templates and patterns for data pipelines using Databricks notebooks and jobs to accelerate project delivery and promote engineering standards • Establish monitoring practices and observability dashboards for data pipelines and platforms so that data quality, latency and availability expectations are continuously met • Guide application and data teams in adopting architectural principles such as modular design, resilience and security by design to reduce operational risks • Collaborate with stakeholder groups to translate business requirements into technical blueprints that clearly define data flows, service responsibilities and integration points • Document architecture decisions, diagrams and data lineage in an accessible form so that engineering and governance teams can maintain and evolve the platforms effectively • Review code, configurations and infrastructure definitions for AWS, Databricks and Palantir Foundry implementations to ensure compliance with enterprise standards • Align architectural roadmaps with company sustainability and social impact goals by supporting data products that improve transparency, efficiency and ethical insights • Participate in hybrid collaboration rituals using digital tools and on site sessions to maintain effective communication, knowledge sharing and continuous improvement. Requirements: • Experience: 6 to 10 Yrs • Possess professional experience of six to ten years working in data engineering or data architecture roles with emphasis on large scale cloud environments • Demonstrate strong practical expertise in core AWS services related to compute, storage, networking and security used for modern data platforms • Exhibit hands on proficiency with Databricks including cluster configuration, notebook development, job orchestration and performance tuning for batch and streaming workloads • Show proven experience implementing data solutions with Palantir Foundry such as data onboarding, ontology design and operational analytic workflows • Apply solid understanding of data warehousing, lakehouse patterns, data governance and metadata management to design consistent enterprise data ecosystems • Use programming skills in languages such as Python and SQL to build, test and troubleshoot complex data transformations and integrations across platforms • Display familiarity with modern DevOps and data operations practices including version control, continuous integration and automated testing for data pipelines Why Should You Apply? • Health Benefits • Referral Program • Excellent growth and advancement opportunities

Requirements

Must-haves

Our client, an IT Services and Consulting company, is looking for a Senior Data Engineer – AWS, Databricks & Palantir Foundry for their Remote location.
Design robust data architectures on AWS that support analytics, reporting and machine learning workloads while ensuring scalability, performance and security compliance
Develop end to end data processing solutions using Databricks that optimize data ingestion, transformation and aggregation for diverse business domains
Coordinate integration of Palantir Foundry platforms with existing enterprise systems to enable unified data views and reusable analytical assets
Implement data modelling practices that standardize reference data, master data and domain specific schemas to support consistent decision making across teams
Optimize cloud resource usage on AWS by refining storage patterns, compute configurations and scheduling strategies to balance cost and performance
Create reusable templates and patterns for data pipelines using Databricks notebooks and jobs to accelerate project delivery and promote engineering standards
Establish monitoring practices and observability dashboards for data pipelines and platforms so that data quality, latency and availability expectations are continuously met
Guide application and data teams in adopting architectural principles such as modular design, resilience and security by design to reduce operational risks
Collaborate with stakeholder groups to translate business requirements into technical blueprints that clearly define data flows, service responsibilities and integration points
Document architecture decisions, diagrams and data lineage in an accessible form so that engineering and governance teams can maintain and evolve the platforms effectively
Review code, configurations and infrastructure definitions for AWS, Databricks and Palantir Foundry implementations to ensure compliance with enterprise standards
Align architectural roadmaps with company sustainability and social impact goals by supporting data products that improve transparency, efficiency and ethical insights
Participate in hybrid collaboration rituals using digital tools and on site sessions to maintain effective communication, knowledge sharing and continuous improvement.
Experience: 6 to 10 Yrs
Possess professional experience of six to ten years working in data engineering or data architecture roles with emphasis on large scale cloud environments
Demonstrate strong practical expertise in core AWS services related to compute, storage, networking and security used for modern data platforms
Exhibit hands on proficiency with Databricks including cluster configuration, notebook development, job orchestration and performance tuning for batch and streaming workloads
Show proven experience implementing data solutions with Palantir Foundry such as data onboarding, ontology design and operational analytic workflows
Apply solid understanding of data warehousing, lakehouse patterns, data governance and metadata management to design consistent enterprise data ecosystems
Use programming skills in languages such as Python and SQL to build, test and troubleshoot complex data transformations and integrations across platforms
Display familiarity with modern DevOps and data operations practices including version control, continuous integration and automated testing for data pipelines
Why Should You Apply?
Referral Program

Nice-to-haves

None flagged.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

awsmachine learningpythonsql

Platform: jobdiva
Effort: medium