Job description

Job Description – Member of Technical Staff (Speech) Location: San Francisco, CA (on-site, 5 days a week, in our San Francisco office) About Artificial Analysis Artificial Analysis is the leading independent AI benchmarking company. We support labs, engineers and enterprises to understand AI capabilities and make critical decisions about their AI strategies. We are the go-to authority for understanding AI, from AI labs and enterprises to media, investors, and policymakers. Our benchmarks don’t just measure the cutting edge of AI, they are actively shaping the frontier. Our benchmarks and analysis are trusted by hundreds of thousands of users and are the go-to reference for leading AI labs including OpenAI, Google, Meta, NVIDIA and Anthropic, and major publications including the Wall Street Journal, Bloomberg, the Financial Times and The Economist. We are a team of 40+, on track to double by end of year, backed by Nat Friedman (GitHub, Meta), Daniel Gross (SSI, Meta), Andrew Ng (Google Brain, DeepLearning.ai deeplearning.ai, Amazon), Adam D’Angelo (Quora, Poe, OpenAI), Clem Delangue (Hugging Face) and other industry leaders. The Opportunity Speech is becoming AI’s next interface: text to speech, speech to text, voice cloning and real-time voice agents are moving from demos into infrastructure, and our speech benchmarks are how the industry tracks who is winning. We’re hiring into our speech pillar to drive that coverage. You’ll build and extend our speech evaluations and arenas, from streaming speech to text and voice cloning to next-generation speech-to-speech intelligence with benchmarks like AgentTalk, and work at a deep technical level with the leading speech AI companies to benchmark their latest models as they launch. What You’ll Do • Benchmark the Speech Frontier: Own coverage across text to speech, speech to text and speech to speech, benchmarking new models and providers as they launch • Design Speech Evaluations: Build and extend the evaluation frameworks, prompt libraries and arenas that reflect how developers and creators actually use speech models, including agentic voice through AgentTalk • Partner with Speech Leaders: Work with the top speech AI companies in the world at a deep technical level to benchmark their models and shape how the industry measures voice • Publish Influential Analysis: Produce the leaderboards, reports and analysis that shape how the industry understands speech AI progress • Drive Product Direction: Shape the roadmap of our speech benchmarking platform together with our engineers and pillar lead • Become AI-Native: Embrace an AI-native workflow, using cutting-edge AI tools to generate leverage in a fast-changing industry and maintain our competitive edge in AI benchmarking What We’re Looking For You should know speech AI from the inside. Backgrounds include: product, research or engineering roles at speech AI companies (e.g. ElevenLabs, Cartesia, Inworld, Sesame, Deepgram, AssemblyAI, Rime or similar), voice teams at larger platforms (e.g. OpenAI, Google, Microsoft), or teams building products on text to speech, speech to text or real-time voice. Required: • 3+ years of professional experience, including at least 1 year working hands-on with speech AI • Strong analytical and critical thinking skills • Proficiency in Python and data analysis • Hands-on familiarity with modern speech models and their evaluation: quality assessment, word error rate and latency measurement, and preference testing • Genuine, demonstrable interest and knowledge of Frontier AI. We want people who have informed opinions about where AI is heading, not just people who use AI tools Why Artificial Analysis? • Shape how AI gets built: The leading AI labs track our benchmarks and use them to guide their development priorities. Your work will directly influence the direction of AI. • Become a world expert in AI: You will evaluate every major model, across every major capability, as they are released. Very few roles offer this breadth of exposure to frontier AI. • Work with the most important players in AI: You’ll manage relationships with teams at the leading AI labs and major enterprises as a trusted, independent voice. • Join at a defining moment: We’re 40+ people, on track to double by end of year, backed by some of the most connected investors in AI. The people who join now will shape the product, the team, and the strategy as we scale. • Competitive compensation including equity

Requirements

Must-haves

Job Description – Member of Technical Staff (Speech)
Location: San Francisco, CA (on-site, 5 days a week, in our San Francisco office)
What You’ll Do
Benchmark the Speech Frontier: Own coverage across text to speech, speech to text and speech to speech, benchmarking new models and providers as they launch
Design Speech Evaluations: Build and extend the evaluation frameworks, prompt libraries and arenas that reflect how developers and creators actually use speech models, including agentic voice through AgentTalk
Partner with Speech Leaders: Work with the top speech AI companies in the world at a deep technical level to benchmark their models and shape how the industry measures voice
Publish Influential Analysis: Produce the leaderboards, reports and analysis that shape how the industry understands speech AI progress
Drive Product Direction: Shape the roadmap of our speech benchmarking platform together with our engineers and pillar lead
Become AI-Native: Embrace an AI-native workflow, using cutting-edge AI tools to generate leverage in a fast-changing industry and maintain our competitive edge in AI benchmarking
What We’re Looking For
You should know speech AI from the inside.
3+ years of professional experience, including at least 1 year working hands-on with speech AI
Strong analytical and critical thinking skills
Proficiency in Python and data analysis
Hands-on familiarity with modern speech models and their evaluation: quality assessment, word error rate and latency measurement, and preference testing
Genuine, demonstrable interest and knowledge of Frontier AI. We want people who have informed opinions about where AI is heading, not just people who use AI tools
Why Artificial Analysis?
Shape how AI gets built: The leading AI labs track our benchmarks and use them to guide their development priorities. Your work will directly influence the direction of AI.
Become a world expert in AI: You will evaluate every major model, across every major capability, as they are released. Very few roles offer this breadth of exposure to frontier AI.
Work with the most important players in AI: You’ll manage relationships with teams at the leading AI labs and major enterprises as a trusted, independent voice.
Competitive compensation including equity

Nice-to-haves

None flagged.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

aigopython

Platform: ashby
Effort: quick