Job description

ACTIVE SECURITY CLEARANCE AT THE TOP SECRET LEVEL IS REQUIRED We are seeking a Lead Data Scientist & Data Engineer to drive the modernization of an enterprise data management ecosystem containing hundreds of thousands of current and historical records for critical defense and intelligence customer missions. In this role, you will design, develop, and deploy scalable data engineering pipelines, advanced Natural Language Processing (NLP) solutions, semantic search frameworks, and automated document tagging capabilities. You will combine hands-on Python development with enterprise platform expertise (such as Databricks and Palantir Foundry) to transform massive volumes of structured and unstructured data into actionable operational intelligence. The annual base salary range for this role is $229,000-$246,000 (USD) , which does not include discretionary bonus compensation or our comprehensive benefits package. Actual compensation offered to the successful candidate may vary from posted hiring range based upon geographic location, work experience, education, and/or skill level, among other things. • Data Engineering Pipelines: Hands-on experience designing, maintaining, and optimizing scalable ETL/ELT pipelines for structured and unstructured enterprise-scale data using Python, SQL, Apache Spark, and Databricks. • NLP & Machine Learning: Demonstrated ability to build, train, and deploy NLP pipelines for document summarization, text classification, clustering, automated tagging, and semantic search into production environments. • Data Quality & Profiling: Expertise conducting data-quality assessments, profiling, cleansing, validation, and anomaly detection to ensure high data integrity across large repositories. • Enterprise Platform Integration: Practical experience developing and deploying data pipelines and analytical workflows within enterprise data/AI platforms like Databricks, Palantir Foundry, or AWS/Azure cloud environments. • Agile & DevSecOps: Experience collaborating in Agile software development environments supporting automated testing, CI/CD, and DevSecOps practices. • Technical Communication & Mentorship: Ability to develop dynamic data visualizations/dashboards, communicate findings to diverse technical and leadership stakeholders, and provide technical guidance to junior engineers. • • Education: Bachelor’s Degree in Data Analytics, Intelligence Studies, Computer Science, Finance/Economics, or a related discipline, PLUS 6 + years of specialized data analytics or financial intelligence experience OR High School Diploma / GED , PLUS 16 + years of hands-on intelligence analysis, financial tracking, and data discovery experience in lieu of a degree. Palantir Foundry Ecosystem: Deep experience with Palantir Foundry, including Pipeline Builder, Code Repositories, custom transformations, and Ontology development. Generative AI & LLMs: Experience with Large Language Models (LLMs), Retrieval-Augmented Generation (RAG architectures), vector databases, and embedding-based search techniques. Advanced ML & Analytics: Hands-on experience with ML frameworks (PyTorch, TensorFlow, scikit-learn) and big data/NoSQL platforms (PySpark, Scala, R, MongoDB). Business Intelligence & Visualization: Experience crafting executive reporting solutions using Tableau, Power BI, Qlik, or Python visualization libraries. Federal & Mission Focus: Direct experience supporting Department of Defense (DoD), Intelligence Community (IC), or secure federal government cloud environments.

Requirements

Must-haves

ACTIVE SECURITY CLEARANCE AT THE TOP SECRET LEVEL IS REQUIRED
Data Engineering Pipelines: Hands-on experience designing, maintaining, and optimizing scalable ETL/ELT pipelines for structured and unstructured enterprise-scale data using Python, SQL, Apache Spark, and Databricks.
NLP & Machine Learning: Demonstrated ability to build, train, and deploy NLP pipelines for document summarization, text classification, clustering, automated tagging, and semantic search into production environments.
Data Quality & Profiling: Expertise conducting data-quality assessments, profiling, cleansing, validation, and anomaly detection to ensure high data integrity across large repositories.
Agile & DevSecOps: Experience collaborating in Agile software development environments supporting automated testing, CI/CD, and DevSecOps practices.
Technical Communication & Mentorship: Ability to develop dynamic data visualizations/dashboards, communicate findings to diverse technical and leadership stakeholders, and provide technical guidance to junior engineers.
Palantir Foundry Ecosystem: Deep experience with Palantir Foundry, including Pipeline Builder, Code Repositories, custom transformations, and Ontology development.
Generative AI & LLMs: Experience with Large Language Models (LLMs), Retrieval-Augmented Generation (RAG architectures), vector databases, and embedding-based search techniques.
Advanced ML & Analytics: Hands-on experience with ML frameworks (PyTorch, TensorFlow, scikit-learn) and big data/NoSQL platforms (PySpark, Scala, R, MongoDB).
Business Intelligence & Visualization: Experience crafting executive reporting solutions using Tableau, Power BI, Qlik, or Python visualization libraries.
Federal & Mission Focus: Direct experience supporting Department of Defense (DoD), Intelligence Community (IC), or secure federal government cloud environments.

Nice-to-haves

None flagged.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

pythonsqlmachine learningaiawsazureagileci/cdml

Platform: agile-ats
Effort: medium