Job description

ABOUT QUALITATE Qualitate is building the AI-native primary intelligence platform for enterprises, investment firms, and asset managers. We automate the entire primary research workflow, from study design and expert recruitment through AI-moderated interviews and insight synthesis - replacing weeks of manual research with hours of high-signal output. Expert research is a broken industry dominated by legacy incumbents with expensive, manual, and slow services. Investment and corporate strategy teams spend hundreds of hours coordinating intermediaries, scheduling expert calls, and synthesizing insights across fragmented providers. The process takes weeks to produce short-lived intelligence. Qualitate transforms this workflow by autonomously conducting expert discussions at scale using AI. Qualitate’s AI Moderator interviews thousands of experts simultaneously each month, capturing forward-looking purchasing plans, competitive displacement, and real-time ROI signals. The platform turns these interviews into quantified, time-series outputs tracking over 10,000 public and private companies, queryable instantly in natural language. Our customers include some of the world’s largest enterprises, hedge funds, PE firms, credit investors, and venture capital firms. THE ROLE We’re looking for a Senior Applied AI Engineer to help build the LLM and retrieval systems at the core of our AI Moderator - the engine that conducts thousands of real-time expert interviews and turns them into structured, queryable intelligence. You’ll work at the intersection of applied research and production engineering: designing agentic pipelines, scaling retrieval over proprietary data, and shipping systems that directly power what our customers see. This is a senior, high-ownership role on a small founding team - your architecture decisions and code will define how the product scales. THE DAY-TO-DAY: - Own the LLM orchestration layer that drives our AI Moderator’s real-time, multi-turn expert interviews, from prompt and context design to model routing and latency optimization. - Instrument evals, monitoring, and guardrails for AI systems running in production, so quality issues surface before customers see them. - Build production APIs and services that turn raw interview data into novel, structured, queryable insights for customers, in partnership with product and data engineering. - Work cross-functionally with our research team to build and ship AI-powered research assets that are surfaced to customers. WHO THIS ROLE IS FOR - 5+ years building and shipping applied AI/ML systems in production, ideally including time at a startup or other fast-paced environment. - Deep, hands-on LLM experience - prompt/system design, retrieval-augmented generation, tool/function calling, and structured output design across multiple model providers (Anthropic, OpenAI, open-weight models). - Strong backend engineering in Python/TypeScript with experience designing and shipping APIs and microservices. - Vector database and search know-how and experience optimizing retrieval quality and latency at scale. - Comfort with unstructured, messy data - audio, transcripts, financial documents - and the pipelines needed to turn it into model-ready input. - A bias for ownership - you can take a problem from rough prototype to production without needing a detailed spec. WHAT SETS YOU APART - Prior experience as an applied AI/ML engineer for a technology startup. - Experience with agentic or multi-turn conversational systems - voice or chat - operating in production with real users. - Experience building or hardening evaluation frameworks (offline + online evals, LLM-as-judge, regression testing on prompt/model changes) for production AI systems. - Familiarity with speech-to-text transcription models (Deepgram, AssemblyAI, Whisper, or similar). - ETL and data engineering experience for dealing with messy unstructured data and setting up automated processes to ingest and transform the data in a relational database - A background in fintech, research, or another AI-driven, data-intensive startup environment. WHAT YOU’LL BUILD IN THE FIRST 90 DAYS - Ship improvements to the LLM orchestration layer powering live AI Moderator interviews, with real customer traffic on the line. - Help improve our downstream LLM processes — retrieval relevance, or post-processing — by iterating labeled data → eval → prompt/code → re-eval. - Partner with the founding team to define what parts of the retrieval and generation stack to build versus buy as usage scales. - Help architect an MCP that sits on top of our proprietary dataset and the relevant tools that make up the orchestration layer COMPENSATION & BENEFITS - Compensation: Competitive base salary + bonus - Equity: Meaningful early-stage grant with upside tied to company growth. - Benefits: Health coverage, flexible PTO, paid holidays. - Location: Full-time, in-person in New York City. WHAT TO EXPECT WORKING AT QUALITATE - Excellence. You’ll be held accountable to an exceptionally high bar and impact - High velocity. Fast paced work environment in terms of growth, decision making, and time to impact - High agency. Ownership over how you work, not just what you work on. We hire people we trust to figure it out. - A builder’s mindset. You’ll create the processes, playbooks, and systems that define how this company scales. - Shared ambition. A small, early team disrupting the multi-billion dollar market research industry with AI. - Joy. We work hard and genuinely enjoy working with each other. Dedication to our work is matched only by our passion for fostering an enjoyable, collaborative work environment. Qualitate is an equal opportunity employer. We encourage candidates of all backgrounds to apply.

Requirements

Must-haves

Own the LLM orchestration layer that drives our AI Moderator’s real-time, multi-turn expert interviews, from prompt and context design to model routing and latency optimization.
Instrument evals, monitoring, and guardrails for AI systems running in production, so quality issues surface before customers see them.
Build production APIs and services that turn raw interview data into novel, structured, queryable insights for customers, in partnership with product and data engineering.
Work cross-functionally with our research team to build and ship AI-powered research assets that are surfaced to customers.
5+ years building and shipping applied AI/ML systems in production, ideally including time at a startup or other fast-paced environment.
Deep, hands-on LLM experience - prompt/system design, retrieval-augmented generation, tool/function calling, and structured output design across multiple model providers (Anthropic, OpenAI, open-weight models).
Strong backend engineering in Python/TypeScript with experience designing and shipping APIs and microservices.
Vector database and search know-how and experience optimizing retrieval quality and latency at scale.
Comfort with unstructured, messy data - audio, transcripts, financial documents - and the pipelines needed to turn it into model-ready input.
A bias for ownership - you can take a problem from rough prototype to production without needing a detailed spec.
Prior experience as an applied AI/ML engineer for a technology startup.
Experience with agentic or multi-turn conversational systems - voice or chat - operating in production with real users.
Experience building or hardening evaluation frameworks (offline + online evals, LLM-as-judge, regression testing on prompt/model changes) for production AI systems.
Familiarity with speech-to-text transcription models (Deepgram, AssemblyAI, Whisper, or similar).
ETL and data engineering experience for dealing with messy unstructured data and setting up automated processes to ingest and transform the data in a relational database
A background in fintech, research, or another AI-driven, data-intensive startup environment.
WHAT YOU’LL BUILD IN THE FIRST 90 DAYS
Ship improvements to the LLM orchestration layer powering live AI Moderator interviews, with real customer traffic on the line.
Help improve our downstream LLM processes — retrieval relevance, or post-processing — by iterating labeled data → eval → prompt/code → re-eval.
Partner with the founding team to define what parts of the retrieval and generation stack to build versus buy as usage scales.
Help architect an MCP that sits on top of our proprietary dataset and the relevant tools that make up the orchestration layer
Excellence. You’ll be held accountable to an exceptionally high bar and impact
High velocity. Fast paced work environment in terms of growth, decision making, and time to impact
High agency. Ownership over how you work, not just what you work on. We hire people we trust to figure it out.

Nice-to-haves

None flagged.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

aillmmlpythontypescript

Platform: ashby
Effort: quick

Benefits

Benefits detected in the listing.

Pto