Job description

Our Client, an IT Services and Consultant company, is looking for a Palantir Foundry Solution Architect – AWS & PySpark for their San Francisco, CA/Hybrid location. Responsibilities: • Design and architect end to end cloud data platforms leveraging AWS Big Data services and PySpark to deliver scalable, resilient and secure data solutions that enable high quality analytics for business stakeholders and downstream systems. • Define robust data ingestion, transformation and storage patterns across structured and semi structured sources using PySpark and AWS native capabilities to optimize performance, reliability and cost efficiency for enterprise workloads. • Develop well structured data models, metadata strategies and data catalog approaches within Palantir Foundry to ensure discoverability, traceability and consistent reuse of curated data assets across multiple business domains. • Establish standardized practices for data pipeline development, testing and deployment in Palantir Foundry and AWS environments to ensure predictable release cycles, reduced incidents and improved operational stability. • Collaborate with product owners, data engineers and analysts to translate business requirements into detailed technical designs and solution blueprints that align with enterprise architecture principles and support strategic outcomes. • Guide teams on best practices for data quality, validation and reconciliation using PySpark frameworks and Foundry capabilities to ensure trusted data sets that underpin critical reporting and decision making processes. • Create architecture diagrams, design documents and technical standards for cloud data solutions to support governance reviews, stakeholder communication and long term maintainability of the data ecosystem. • Assess existing data workloads and pipelines in AWS and Palantir Foundry to identify optimization opportunities, streamline resource usage and improve throughput while maintaining compliance and security expectations. • Coordinate closely with cybersecurity and compliance teams to embed access controls, encryption practices and auditing features into data platform designs so that regulatory and company policy requirements are consistently met. • Support incident analysis and problem resolution for complex data pipeline issues by conducting root cause assessments, implementing corrective patterns and updating reference designs to prevent recurrence. • Engage with cross functional teams to plan migration or modernization of legacy data solutions into AWS Big Data and Palantir Foundry, ensuring minimal disruption, clear cutover strategies and robust validation. • Document repeatable solution patterns and reference implementations for AWS Big Data, PySpark and Palantir Foundry so that project teams can adopt proven approaches and accelerate delivery timelines. • Influence technology roadmaps and data platform evolution by providing informed perspectives on emerging AWS Big Data capabilities, PySpark enhancements and Foundry features that can expand analytical reach and societal impact. Requirements: • PySpark, AWS Big Data, Palantir Foundry • Possess strong hands on experience in architecting large scale data platforms on AWS Big Data services including core storage, processing and orchestration components to support complex enterprise data needs. • Demonstrate advanced proficiency in PySpark for building high performance data transformation pipelines, reusable frameworks and batch or near real time processing flows at significant data volumes. • Bring deep practical exposure to designing and implementing solutions in Palantir Foundry including ontology design, transformation logic, governance configuration and collaborative analytical workflows. • Hold solid knowledge of data warehousing, data lake concepts and modern lakehouse practices to design models that support both operational reporting and advanced analytics use cases. • Show strong understanding of data security, privacy principles and compliance expectations in cloud environments so that architectures meet regulatory requirements while enabling responsible data usage. • Exhibit experience working in hybrid work settings and coordinating effectively with distributed teams during day shifts to ensure continuous progress on design, implementation and review activities. • Display capability to communicate complex architectural decisions to technical and non technical stakeholders through clear documentation, modeling artifacts and structured discussions that support alignment and adoption. • Years of Experience: 13.00 Years of Experience Why Should You Apply? • Health Benefits • Referral Program • Excellent growth and advancement opportunities

Requirements

Must-haves

Our Client, an IT Services and Consultant company, is looking for a Palantir Foundry Solution Architect – AWS & PySpark for their San Francisco, CA/Hybrid location.
Establish standardized practices for data pipeline development, testing and deployment in Palantir Foundry and AWS environments to ensure predictable release cycles, reduced incidents and improved operational stability.
Guide teams on best practices for data quality, validation and reconciliation using PySpark frameworks and Foundry capabilities to ensure trusted data sets that underpin critical reporting and decision making processes.
Create architecture diagrams, design documents and technical standards for cloud data solutions to support governance reviews, stakeholder communication and long term maintainability of the data ecosystem.
Assess existing data workloads and pipelines in AWS and Palantir Foundry to identify optimization opportunities, streamline resource usage and improve throughput while maintaining compliance and security expectations.
Support incident analysis and problem resolution for complex data pipeline issues by conducting root cause assessments, implementing corrective patterns and updating reference designs to prevent recurrence.
Engage with cross functional teams to plan migration or modernization of legacy data solutions into AWS Big Data and Palantir Foundry, ensuring minimal disruption, clear cutover strategies and robust validation.
Document repeatable solution patterns and reference implementations for AWS Big Data, PySpark and Palantir Foundry so that project teams can adopt proven approaches and accelerate delivery timelines.
PySpark, AWS Big Data, Palantir Foundry
Possess strong hands on experience in architecting large scale data platforms on AWS Big Data services including core storage, processing and orchestration components to support complex enterprise data needs.
Demonstrate advanced proficiency in PySpark for building high performance data transformation pipelines, reusable frameworks and batch or near real time processing flows at significant data volumes.
Bring deep practical exposure to designing and implementing solutions in Palantir Foundry including ontology design, transformation logic, governance configuration and collaborative analytical workflows.
Hold solid knowledge of data warehousing, data lake concepts and modern lakehouse practices to design models that support both operational reporting and advanced analytics use cases.
Show strong understanding of data security, privacy principles and compliance expectations in cloud environments so that architectures meet regulatory requirements while enabling responsible data usage.
Exhibit experience working in hybrid work settings and coordinating effectively with distributed teams during day shifts to ensure continuous progress on design, implementation and review activities.
Years of Experience: 13.00 Years of Experience
Why Should You Apply?
Referral Program
Excellent growth and advancement opportunities

Nice-to-haves

None flagged.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

aws

Platform: jobdiva
Effort: medium