Job description

Kensho is S&P Global’s hub for AI innovation and transformation. With expertise in machine learning, natural language processing, and data discovery, we develop and deploy novel solutions to innovate and drive progress at S&P Global and its customers worldwide. Kensho s solutions and research focus on business and financial generative AI applications, agents, data retrieval APIs, data extraction, and much more. At Kensho, we hire talented people and give them the autonomy and support needed to build amazing technology and products. We collaborate using our teammates diverse perspectives to solve hard problems. Our communication with one another is open, honest, and efficient. We dedicate time and resources to explore new ideas, but always rooted in engineering best practices. As a result, we can innovate rapidly to produce technology that is scalable, robust, and useful. The DRIVE Team at Kensho is focused on designing and deploying production-grade machine learning systems that power our next-generation agentic search pipelines. We specialize in building robust retrieval systems, scalable embedding infrastructure, and tightly integrated LLM pipelines that leverage unstructured data sources. Our mission is to make complex unstructured data easily discoverable and actionable by building intelligent, retrieval-driven systems that enhance enterprise search, question answering, deep research, report generation, and knowledge discovery experiences across S&P Global platforms. We are seeking a mid-level Machine Learning Engineer to help develop and scale RAG systems across the company. This is a hands-on, full-lifecycle ML role with a strong emphasis on retrieval models, LLM orchestration, and system-level thinking. Kensho states that the anticipated base salary range for the position is 140k - 180k. In addition, this role is eligible for an annual incentive bonus and equity plans. At Kensho, it is not typical for an individual to be hired at or near the top of the range for their role and compensation decisions are dependent on the facts and circumstances of each case. What You’ll Do: Design and implement end-to-end RAG pipelines that integrate proprietary chunking algorithms, embedding models, vector databases, and data retrieval agents Build and optimize retrieval systems over large-scale proprietary datasets using advanced embedding techniques Develop LLM-based solutions that orchestrate retrieval, generation, and ranking to deliver high-quality, context-aware responses Investigate and solve challenges in vector search, chunking and indexing strategies, unstructured data retrieval evaluation, and GraphRAG Work closely with Product and Design teams to build ML-based solutions that enhance user experiences and meet business objectives Collaborate closely with the ML Operations team to create automated solutions for managing the entire ML systems lifecycle, from initial technical design to seamless implementation Who You ll Need: Bachelor s degree or higher in Computer Science, Engineering, or a related field. 3 years of significant, hands-on industry experience with machine learning, natural language processing (NLP), information retrieval systems and large-scale text processing, including designing, shipping, and maintaining production systems Strong programming skills in Python, with a working knowledge of data processing tools and ML frameworks such as PyTorch, Transformers, and HuggingFace Experience working with machine learning libraries/frameworks for Large Language Model (LLM) orchestration, such as Langchain, LLamaIndex, etc. Proven experience building ML pipelines for data processing, training, inference, maintenance, evaluation, versioning, and experimentation. Experience working with vector databases (e.g., PostgreSQL/PGVector, OpenSearch, Pinecone) and understanding of similarity search techniques and vector indexing algorithms Demonstrated effective coding, documentation, collaboration, and communication habits Strong problem-solving skills and a proactive approach to addressing challenges Ability to adapt to a fast-paced and dynamic work environment Technologies We Love: ML: PyTorch, Transformers, HuggingFace, LangChain Tools/Toolkits: Claude Code, Weights & Biases, OpenSearch, PostgreSQL/PGVector, LiteLLM Techniques : Agentic Search, Prompt Engineering, Information Retrieval, Data Embedding, AI agent evaluation Deployment: Airflow, Docker, Kubernetes, Jenkins, AWS, Github Action At Kensho, we pride ourselves on providing top-of-market benefits, including:   Medical, Dental, and Vision insurance   100% company paid premiums  Unlimited Paid Time Off  26 weeks of 100% paid Parental Leave (paternity and maternity)  401(k) plan with 6% employer matching  Generous company matching on donations to non-profit charities  Up to $20,000 tuition assistance toward degree programs, plus up to $4,000/year for ongoing professional education such as industry conferences  Plentiful snacks, drinks, and regularly catered lunches  Dog-friendly office (CAM office)  Bike sharing program memberships  Compassion leave and elder care leave  Mentoring and additional learning opportunities  Opportunity to expand professional network and participate in conferences and events  Recruitment Fraud Alert: If you receive an email from a spglobalind.com domain or any other regionally based domains, it is a scam and should be reported to reportfraud spglobal.com . S&P Global never requires any candidate to pay money for job applications, interviews, offer letters, “pre-employment training” or for equipment/delivery of equipment. Stay informed and protect yourself from recruitment fraud by reviewing our guidelines, fraudulent domains, and how to report suspicious activity here . We are an equal opportunity employer that welcomes future Kenshins with all experiences and perspectives. Kensho is headquartered in Cambridge, MA, with an additional office location in New York City. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, or national origin.

Requirements

Must-haves

Design and implement end-to-end RAG pipelines that integrate proprietary chunking algorithms, embedding models, vector databases, and data retrieval agents
Build and optimize retrieval systems over large-scale proprietary datasets using advanced embedding techniques
Develop LLM-based solutions that orchestrate retrieval, generation, and ranking to deliver high-quality, context-aware responses
Investigate and solve challenges in vector search, chunking and indexing strategies, unstructured data retrieval evaluation, and GraphRAG
Work closely with Product and Design teams to build ML-based solutions that enhance user experiences and meet business objectives
Collaborate closely with the ML Operations team to create automated solutions for managing the entire ML systems lifecycle, from initial technical design to seamless implementation
Bachelor s degree or higher in Computer Science, Engineering, or a related field.
Strong programming skills in Python, with a working knowledge of data processing tools and ML frameworks such as PyTorch, Transformers, and HuggingFace
Experience working with machine learning libraries/frameworks for Large Language Model (LLM) orchestration, such as Langchain, LLamaIndex, etc.
Proven experience building ML pipelines for data processing, training, inference, maintenance, evaluation, versioning, and experimentation.
Experience working with vector databases (e.g., PostgreSQL/PGVector, OpenSearch, Pinecone) and understanding of similarity search techniques and vector indexing algorithms
Demonstrated effective coding, documentation, collaboration, and communication habits
Strong problem-solving skills and a proactive approach to addressing challenges
Ability to adapt to a fast-paced and dynamic work environment
ML: PyTorch, Transformers, HuggingFace, LangChain
Tools/Toolkits: Claude Code, Weights & Biases, OpenSearch, PostgreSQL/PGVector, LiteLLM
Techniques : Agentic Search, Prompt Engineering, Information Retrieval, Data Embedding, AI agent evaluation
Deployment: Airflow, Docker, Kubernetes, Jenkins, AWS, Github Action
At Kensho, we pride ourselves on providing top-of-market benefits, including:
100% company paid premiums
Generous company matching on donations to non-profit charities
Up to $20,000 tuition assistance toward degree programs, plus up to $4,000/year for ongoing professional education such as industry conferences
Plentiful snacks, drinks, and regularly catered lunches
Dog-friendly office (CAM office)

Nice-to-haves

None flagged.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

aimachine learningllmmlpythondockerkubernetesaws

Platform: workday
Effort: long

Benefits

Benefits detected in the listing.

Dental

Vision

401(K)

Paid Time Off

Parental Leave