Job description

This role focuses on leading a team working on developing, carrying-out, interpreting, and communicating pre- and post-ship evaluations of the safety of Apple Intelligence features. This team is also responsible for producing safety evaluations that uphold Apple’s Responsible AI values requires thoughtful data sampling, creation, and curation for evaluation datasets; high quality, detailed annotations and careful auto-grading to assess feature performance; and mindful analysis to understand what the evaluation means for the user experience. MS, or PhD in Computer Science, Machine Learning, Statistics, or related fields; or an equivalent qualification acquired through other avenues. Experience working with generative models for evaluation and/or product development, and up-to-date knowledge of common challenges and failures. Strong engineering skills and experience in writing production-quality code in Python. Deep experience in foundation model-based AI programming (i.e.: using DSPy for optimizing foundation model prompts, for example) and a drive to innovate in this space. Experience working with noisy, crowd-based data labels and human evaluations. Publication record in relevant conferences (e.g., NeurIPS, ICML, ICLR, EMNLP, etc.) Experience working on Responsible AI and AI Safety Strong organizational and operational skills working with large, multi-functional, and diverse teams. Curiosity about fairness and bias in generative AI systems, and a strong desire to help make the technology more equitable.

Requirements

Must-haves

MS, or PhD in Computer Science, Machine Learning, Statistics, or related fields; or an equivalent qualification acquired through other avenues.
Experience working with generative models for evaluation and/or product development, and up-to-date knowledge of common challenges and failures.
Strong engineering skills and experience in writing production-quality code in Python.
Deep experience in foundation model-based AI programming (i.e.: using DSPy for optimizing foundation model prompts, for example) and a drive to innovate in this space.
Experience working with noisy, crowd-based data labels and human evaluations.
Publication record in relevant conferences (e.g., NeurIPS, ICML, ICLR, EMNLP, etc.)
Experience working on Responsible AI and AI Safety
Strong organizational and operational skills working with large, multi-functional, and diverse teams.
Curiosity about fairness and bias in generative AI systems, and a strong desire to help make the technology more equitable.

Nice-to-haves

None flagged.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

aimachine learningpython

Platform: apple
Effort: medium