ABOUT PLATO
Plato is an applied research lab building the foundational infrastructure to train specialized AI agents.
We turn real-world data streams into high-fidelity simulated environments that generate the training signal needed to make capable models. Our work supports frontier labs, hyperscalers, and enterprises building AI systems for complex, high-stakes work.
Today, only a handful of players can train models for capable work. Compute and algorithms are rapidly commoditizing, but reinforcement learning data remains the bottleneck. Plato is changing that by automatically scaling training environments from proprietary real-world data.
WHY APPLY
- Work with the frontier: Build safety environments and benchmarks that research teams at leading AI labs can use to measure emerging capabilities and misaligned behavior.
- Real ownership: Help set Plato’s safety evaluation agenda, from the research questions worth testing to the evidence we share with lab partners.
- Research to production: Work with engineers to turn experimental designs into runnable environments, with a short path from an idea to an evaluation labs can use.
THE ROLE
Plato is hiring an individual contributor researcher to design safety environments and benchmarks, starting with evaluations that measure how well AI models can improve themselves and other models.
You will establish a new research direction at Plato, designing safety-relevant benchmarks and working with engineers to turn them into runnable environments. Your initial focus will be measuring AI R&D capabilities, including experiment design, model training, and research replication.
You will also investigate reward hacking, sabotage, deception, and other misaligned behavior. The work spans research design, empirical evaluation, and failure analysis, with substantial ownership of the research agenda. Some environments may also be used in training to reduce the behaviors they reveal.
WHAT YOU’LL DO
- Design Safety Benchmarks: Measure AI research automation and self-improvement. Define tasks, baselines, budgets, and success criteria that separate meaningful progress from superficial completion.
- Build Safety Environments: Design environments that expose reward hacking, sabotage, deception, and failures to follow intended constraints.
- Hold the Evaluation Bar: Investigate contamination, grader weaknesses, task ambiguity, and shortcuts that could make a result misleading.
- Track Frontier Capabilities: Run experiments with current models, inspect trajectories and failures, and measure how capabilities change across models and over time.
- Work with Engineering: Turn research designs into reproducible environments with clear scoring, useful instrumentation, and documented limitations.
- Share the Findings: Translate results into benchmark releases, research reports, and concrete recommendations for lab partners.
REQUIRED QUALIFICATIONS
- Recent Research: Hands-on work in frontier-model evaluation, benchmarks, or AI safety in the past one to two years.
- Research Track Record: Meaningful contributions to papers, released benchmarks, public reports, or comparable empirical work. We care about the quality of your work and your contribution more than credentials or years of experience.
- Current Model Knowledge: Practical familiarity with current frontier models and a specific, evidence-based view of their capabilities, limitations, and likely development.
- Experimental Judgment: You can explain what an evaluation measures, how it could be gamed, and which conclusions its results do and do not support.
- Research Ownership: Ability to turn an ambiguous question into a concrete experimental design and collaborate closely with engineers to implement it.
- Safety Focus: Interest in measuring safety-relevant capabilities and behavior, including the tradeoffs when evaluation environments are also used for training.
- Bay Area Based: Based in the San Francisco Bay Area, or willing to relocate.
PREFERRED QUALIFICATIONS
- Designed and released a benchmark, particularly for AI R&D, long-horizon agents, dangerous capabilities, or AI control. Relevant reference points include RE-Bench, MLE-bench, and PaperBench.
- Experience in an independent evaluation or safety organization, a frontier lab safety team, or pre-deployment testing with labs or government evaluation bodies.
- Direct research on models conducting ML experiments, training or improving other models, replicating papers, or completing extended research tasks.
- A software engineering background or programming skills that let you implement environments, evaluation harnesses, and analysis tools yourself.
- Experience publishing rigorous empirical research, including work at major machine-learning venues.