Job description

The Sensing and Connectivity team is looking for a manager to lead a talented group of engineers who evaluate the algorithms and AI models behind our location context features, and who also own the quality engineering that ships them: test automation, software testing, and continuous integration. We work in a fast-paced environment that depends on a tight relationship between evaluation and development. You'll evaluate probabilistic, on-device models, defining what "correct" means where ground truth is scarce and real-world behavior can't be fully simulated. In this role you will: - Own the model evaluation strategy for our ML models and other algorithms, including benchmarks, evaluation protocols, and validation of training data, features, and labels, along with the quality strategy for the end-to-end experiences they power. - Set the technical direction for how the team evaluates these models. That means challenging design and implementation, turning assessments of model outputs into prioritized algorithm improvements, and holding the team to the same rigor in its own analysis and conclusions. - Lead the team in building the automation, continuous integration, analytics, and monitoring that qualify features, measure performance and system memory usage, and track quality across releases. - Identify the KPIs that define project success, build the dashboards behind them, and represent the team's results in user study, code, and data reviews as well as to engineering and executive audiences. - Drive bug resolution across the broader software engineering organization and with partner teams across Apple, escalating and negotiating as needed to hold the quality bar. - Work closely with development, program management, and cross-functional partners, from upstream hardware to downstream apps, to drive excellent end-to-end quality aligned with release schedules. - Set a vision for the team's future direction, identify new opportunities, and build stakeholder buy-in to pursue them. - Hire, coach, mentor, and grow the team, building on individual strengths and creating meaningful career growth. BS, MS, or PhD in Computer Science / Electrical Engineering or equivalent 5+ years of relevant industry experience 2+ years leading and managing an engineering team, with a track record of growing engineers' skill sets and creating meaningful career growth Experience evaluating AI/ML models: crafting benchmarks and evaluation protocols, and performing statistical analysis Proficient in designing AI-assisted workflows, using GenAI to automate and accelerate testing, debugging, and metrics analysis Hands-on engineering background, with enough software development experience to set technical direction and critically evaluate the team's work Experience evaluating foundation models or LLMs, for example building eval harnesses, golden datasets, LLM-as-judge pipelines, or human-in-the-loop evaluation Experience with sensor, time-series, or on-device ML models, and with the data pipelines behind them Experience building or leading automation, test infrastructure, and CI systems Experience working at the intersection of hardware and software Ability to work in a dynamic environment with shifting priorities and schedules, with excellent analytical, problem-solving, and communication skills

Requirements

Must-haves

Lead the team in building the automation, continuous integration, analytics, and monitoring that qualify features, measure performance and system memory usage, and track quality across releases.
Identify the KPIs that define project success, build the dashboards behind them, and represent the team's results in user study, code, and data reviews as well as to engineering and executive audiences.
Drive bug resolution across the broader software engineering organization and with partner teams across Apple, escalating and negotiating as needed to hold the quality bar.
Work closely with development, program management, and cross-functional partners, from upstream hardware to downstream apps, to drive excellent end-to-end quality aligned with release schedules.
Set a vision for the team's future direction, identify new opportunities, and build stakeholder buy-in to pursue them.
Hire, coach, mentor, and grow the team, building on individual strengths and creating meaningful career growth.
BS, MS, or PhD in Computer Science / Electrical Engineering or equivalent
5+ years of relevant industry experience
2+ years leading and managing an engineering team, with a track record of growing engineers' skill sets and creating meaningful career growth
Experience evaluating AI/ML models: crafting benchmarks and evaluation protocols, and performing statistical analysis
Proficient in designing AI-assisted workflows, using GenAI to automate and accelerate testing, debugging, and metrics analysis
Hands-on engineering background, with enough software development experience to set technical direction and critically evaluate the team's work
Experience evaluating foundation models or LLMs, for example building eval harnesses, golden datasets, LLM-as-judge pipelines, or human-in-the-loop evaluation
Experience with sensor, time-series, or on-device ML models, and with the data pipelines behind them
Experience building or leading automation, test infrastructure, and CI systems
Experience working at the intersection of hardware and software
Ability to work in a dynamic environment with shifting priorities and schedules, with excellent analytical, problem-solving, and communication skills

Nice-to-haves

None flagged.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

aimlllm

Platform: apple
Effort: medium

Benefits

Benefits detected in the listing.

Vision