Job description

Overview Windows + Devices powers experiences for more than a billion users worldwide, building the platforms and products that enable Microsoft’s Windows and Surface ecosystems. The Windows Engineering System is the foundation that enables engineers to design, build, validate, and ship Windows at massive scale with high quality and security.Windows Engineering Systems currently runs 30 million machine-years of testing every night , providing critical quality signals to more than 5,000 developers across the organization. As engineering velocity continues to accelerate with AI-assisted development, we have an opportunity to fundamentally transform how software validation, failure diagnosis, and quality engineering are performed. We are looking for a Senior Product Manager to lead the Windows Quality Signal AI strategy. In this role, you will define and deliver AI-powered experiences that help engineers diagnose failures faster, surface likely root causes, improve software quality, and strengthen validation signals across Windows engineering.Your work will directly improve Windows quality, engineering productivity, and the effectiveness of validation systems used by thousands of developers across Microsoft. Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond. Responsibilities • Define and drive the vision, strategy, and roadmap for AI-assisted test failure analysis across Windows Engineering Systems. • Build Copilot- and MCP-powered experiences that analyze test artifacts, logs, telemetry, and error signals to surface actionable root-cause insights directly within engineering workflows. • Transform test failure analysis from a fragmented, manual investigation process into a streamlined experience that brings together relevant data, historical signals, and quality insights across engineering systems. • Improve developer productivity by reducing investigation time, minimizing context switching, and integrating AI-assisted diagnostics into the engineering inner loop. • Establish success metrics, AI evaluation frameworks, and quality measures focused on adoption, resolution effectiveness, signal quality, and reductions in no-repro outcomes. • Partner across Engineering Systems, Quality Engineering, Azure DevOps, and Copilot platform teams to deliver trusted AI solutions that improve validation quality and release confidence Qualifications Required Qualifications: • Bachelor's Degree AND 5+ years experience in product/service/program management or software development. • OR equivalent experience. Additional or preferred qualifications Other Requirements: Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings: • Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter. Preferred Qualifications: • Bachelor's Degree AND 8+ years of experience in product management, program management, software engineering, or a related technical field, or equivalent experience. • Experience delivering software products, engineering systems, developer tools, or platform services. • Experience working on software quality, testing, validation, debugging, CI/CD systems, engineering infrastructure, or developer productivity solutions. • Demonstrated ability to define product strategy and execute across multiple engineering organizations. • Strong analytical skills and a data-driven approach to decision-making. • Strong written, verbal, and executive communication skills. • Experience building AI-powered products, copilots, agents, or LLM-based solutions. • Experience defining AI evaluation frameworks, benchmarks, and quality measurement systems. • Familiarity with Azure DevOps, CI/CD pipelines, test infrastructure, and engineering workflows. • Experience designing or operating large-scale distributed systems and cloud services. • Understanding of software validation, failure analysis, test automation, and debugging methodologies. • Proven track record of leading ambiguous, cross-organizational initiatives that deliver measurable business impact. #W+DJOBS #windowses #esjobs Product Management IC4 - The typical base pay range for this role across the U.S. is USD $119,800 - $234,700 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $160,200 - $261,000 per year. Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: careers.microsoft.com This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled. Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.

Requirements

Must-haves

Define and drive the vision, strategy, and roadmap for AI-assisted test failure analysis across Windows Engineering Systems.
Build Copilot- and MCP-powered experiences that analyze test artifacts, logs, telemetry, and error signals to surface actionable root-cause insights directly within engineering workflows.
Transform test failure analysis from a fragmented, manual investigation process into a streamlined experience that brings together relevant data, historical signals, and quality insights across engineering systems.
Improve developer productivity by reducing investigation time, minimizing context switching, and integrating AI-assisted diagnostics into the engineering inner loop.
Establish success metrics, AI evaluation frameworks, and quality measures focused on adoption, resolution effectiveness, signal quality, and reductions in no-repro outcomes.
Partner across Engineering Systems, Quality Engineering, Azure DevOps, and Copilot platform teams to deliver trusted AI solutions that improve validation quality and release confidence
Bachelor's Degree AND 5+ years experience in product/service/program management or software development. • OR equivalent experience.
Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
Bachelor's Degree AND 8+ years of experience in product management, program management, software engineering, or a related technical field, or equivalent experience.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

Nice-to-haves

Additional or preferred qualifications
Experience delivering software products, engineering systems, developer tools, or platform services.
Experience working on software quality, testing, validation, debugging, CI/CD systems, engineering infrastructure, or developer productivity solutions.
Demonstrated ability to define product strategy and execute across multiple engineering organizations.
Strong analytical skills and a data-driven approach to decision-making.
Strong written, verbal, and executive communication skills.
Experience building AI-powered products, copilots, agents, or LLM-based solutions.
Experience defining AI evaluation frameworks, benchmarks, and quality measurement systems.
Familiarity with Azure DevOps, CI/CD pipelines, test infrastructure, and engineering workflows.
Experience designing or operating large-scale distributed systems and cloud services.
Understanding of software validation, failure analysis, test automation, and debugging methodologies.
Proven track record of leading ambiguous, cross-organizational initiatives that deliver measurable business impact.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

aiazureci/cdllm

Platform: eightfold
Effort: medium

Benefits

Benefits detected in the listing.

Vision