Job description

Location: San Francisco, CA (on-site 5 days/week) About The Center for AI Safety (CAIS) The Center for AI Safety (CAIS) is a leading research and advocacy organization focused on mitigating societal-scale risks from AI. Some of our past achievements include: releasing the most widely used measure of AI capabilities used by all major AI companies, running a large compute cluster to facilitate AI safety research which has been cited over 16,000 times , and publishing a global statement on AI Risk signed by Geoffrey Hinton, Yoshua Bengio and top AI CEOs. What distinguishes us is what we choose to work on. Our work is aimed at reducing real-world risks from advanced AI systems. We deliberately pursue research directions that the field is not yet paying attention to, and we move on once the rest of the field catches up. Our focus is on problems that are both highly important and highly neglected—and our track record is built on getting to them first. In 2022–2023 , we focused on AI honesty, robustness, transparency, and trojan/backdoor behaviors. In 2023–2024 , we turned to malicious use and weaponization capabilities, introducing the first state-of-the-art benchmarks for measuring it. More recently, we've been working on AI value systems and the functional well-being of AI systems. This is a research philosophy as opposed to a fixed agenda: we go where the important, unworked problems are. Because our work tends to be timely and to open up territory rather than crowd into it, our papers have repeatedly gone on to become widely cited and to set the standard for underexplored research areas. Our work is regularly used by AI safety institutes and frontier AI labs, and they have shaped real policy outcomes, including being presented directly to senators and policymakers. The Role As a Research Engineer (RE) or Research Scientist (RS) at CAIS, you'll lead and execute high-impact research that advances the safety and reliability of frontier AI systems. This is the general posting for both roles—if you're interested in either, apply here, and we'll determine which is the better fit based on our judgment during the process. You will design and run experiments on large language models, build the tooling to train and evaluate models at scale, and turn results into publishable research. You'll work closely with CAIS researchers and external academic and commercial partners, using our compute cluster to run large-scale training and evaluation. Our work centers on empirical deep learning research with large language models and/or multimodal models. If your background is primarily theoretical, this role may not be a good fit. Research directions at CAIS are set by our Research Director, who selects and prioritizes projects for their importance, neglectedness, and timeliness — a major reason our work has been so impactful. In practice, this means that you will consistently be working on interesting, high-impact problems, with substantial freedom in how you pursue them: designing and running experiments, iterating, and chasing the threads you find most promising. What You'll Do Own research experiments end-to-end. Train and fine-tune large transformer models across domains. Build and maintain datasets and benchmarks. Run distributed training and evaluation at scale. Write and ship research, collaborating with co-authors, and supporting submissions of papers to top conferences. Collaborate with researchers and external partners while contributing to shared research direction and responding quickly in research cycles. Support research infrastructure as needed, such as internal tooling, documentation, and reproducibility practices for the team. Mentor, guide, and support others on the team. What We're Looking For Have co-authored multiple papers published at a top ML conference venues (e.g., NeurIPS, ICML, ICLR, ACL, CVPR). Workshop papers are considered, though peer-reviewed conference publications are strongly preferred. Publications in journals such as IEEE or Springer Nature are typically given less weight. Alternatively, have made meaningful research contributions at a leading AI lab. Are a current PhD student or researcher in machine learning or a related field. Exceptional candidates with a strong publication record may be considered regardless of degree level. Have a track record of empirical research in AI or ML, particularly in AI safety-relevant areas (e.g. adversarial robustness, calibration, benchmarking). We weight empirical research heavily; candidates with primarily theoretical backgrounds are generally not a strong fit. Are able to read an ML paper, understand the key result, and understand how it fits into the broader literature. Are comfortable setting up, launching, and debugging ML experiments. Are familiar with relevant frameworks and libraries (e.g., PyTorch). Communicate clearly and promptly, and take ownership of your part of a project. Salary Range $140,000 - $200,000 annually, based on experience. Benefits Health insurance for you and your dependents 401K plan + 4% matching Unlimited PTO Lunch and dinner at the office Annual Professional Development Stipend Access to some of the top talent working on technical and conceptual research in AI safety Referral Program Know someone who could be a great fit for this role? Submit their details through our Referral Form . If we end up hiring your referral, you’ll receive a $1,500 bonus once they’ve been with CAIS for 90 days. Equal Opportunity Employer The Center for AI Safety is an Equal Opportunity Employer. We consider all qualified applicants without regard to race, color, religion, sex, sexual orientation, gender identity or expression, national origin, ancestry, age, disability, medical condition, marital status, military or veteran status, or any other protected status in accordance with applicable federal, state, and local laws. In alignment with the San Francisco Fair Chance Ordinance, we will consider qualified applicants with arrest and conviction records for employment. If you require a reasonable accommodation during the application or interview process, please contact contact@safe.ai. We value diversity and encourage individuals from all backgrounds to apply.

Requirements

Must-haves

Location: San Francisco, CA (on-site 5 days/week)
About The Center for AI Safety (CAIS)
In 2022–2023 , we focused on AI honesty, robustness, transparency, and trojan/backdoor behaviors.
In 2023–2024 , we turned to malicious use and weaponization capabilities, introducing the first state-of-the-art benchmarks for measuring it.
More recently, we've been working on AI value systems and the functional well-being of AI systems.
Our work centers on empirical deep learning research with large language models and/or multimodal models. If your background is primarily theoretical, this role may not be a good fit.
Own research experiments end-to-end.
Train and fine-tune large transformer models across domains.
Build and maintain datasets and benchmarks.
Run distributed training and evaluation at scale.
Write and ship research, collaborating with co-authors, and supporting submissions of papers to top conferences.
Collaborate with researchers and external partners while contributing to shared research direction and responding quickly in research cycles.
Support research infrastructure as needed, such as internal tooling, documentation, and reproducibility practices for the team.
Mentor, guide, and support others on the team.
Are a current PhD student or researcher in machine learning or a related field. Exceptional candidates with a strong publication record may be considered regardless of degree level.
Are able to read an ML paper, understand the key result, and understand how it fits into the broader literature.
Are comfortable setting up, launching, and debugging ML experiments.
Are familiar with relevant frameworks and libraries (e.g., PyTorch).
Communicate clearly and promptly, and take ownership of your part of a project.
$140,000 - $200,000 annually, based on experience.

Nice-to-haves

None flagged.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

airestgomlmachine learning

Platform: greenhouse
Effort: quick

Benefits

Benefits detected in the listing.

Health Insurance

401k

Pto

Professional Development