Job description

Overview The rapid evolution of AI is creating new possibilities to automate complex work and scale what teams can accomplish. Realizing that potential requires someone who can see where AI can fundamentally change how work gets done—and move quickly from possibility to practical impact. The Copilot Marketing team is looking for a highly technical, AI-forward Senior Technical Product Marketing Manager - AI Automation to reinvent how we evaluate, validate, and improve Microsoft Copilot. Blending product development, technical problem-solving, and a builder’s approach to AI automation, you will create agents, applications, scripts, and workflows that turn manually, fragmented evaluation processes into rigorous, repeatable automated systems. You will partner across teams to automate every aspect of the workflow – from our evaluation methodology end-to-end, to frequently surfacing actionable product and competitive insights, to ultimately how we equip our sellers to clearly demonstrate why customers should bet on Copilot. This role is for a hands-on builder who understands product and platforms , gets deep into how technology works, and moves quickly from experimentation to implementation. If you thrive on solving ambiguous problems, staying ahead of competitive offerings, and navigating and collaborating across a complex matrix of teams and tools to make ideas real, this role is for you. Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond. In alignment with our Microsoft values, we are committed to cultivating an inclusive work environment for all employees to positively impact our culture every day. Responsibilities The rapid evolution of AI is creating new possibilities to automate complex work and scale what teams can accomplish. Realizing that potential requires someone who can see where AI can fundamentally change how work gets done—and move quickly from possibility to practical impact. • Build AI agents, applications, and automated workflows that accelerate product testing, competitive evaluation, validation, and readiness assessment. • Orchestrate the automation of end-to-end evaluation process—from test setup and execution through scoring, validation, analysis, and reporting. • Detect failures, regressions, inconsistencies, and anomalies at scale, with clear mechanisms for human review where judgment matters. • Prototype agentic evaluation approaches—including agents that test other agents—and rapidly move the strongest concepts into production workflows. • Turn evaluation signals into actionable product and competitive insights that shape prioritization, readiness, differentiation, and customer experience. • Scale AI automation across the organization through reusable tools, templates, documentation, responsible practices, and hands-on technical leadership. • Embody our Culture   & Values . Qualifications Required/minimum qualifications • Master's Degree in in Marketing, Computer Science, Business or related field AND 3+ years experience in business OR Bachelor's Degree in Marketing, Computer Science, Business or related field AND 5+ years experience in business OR equivalent experience. Additional or preferred qualifications • Bachelor’s Degree AND 8+ years experience, OR Master’s Degree AND 6+ years experience • Experience building AI agents, agentic workflows, or prototypes using code, APIs, structured data, JSON, and developer tools. • Strong understanding of generative AI, LLMs, agents, and AI product experiences. • Experience with AI evaluation, quality measurement, experimentation, or automated testing, including concepts such as automated graders, human evaluation, judge models, benchmarks, and regression testing. • Strong analytical skills and the ability to design systematic approaches to testing and validation. • Excellent written and verbal communication skills, with the ability to make complex technical concepts clear to non-technical audiences. • Demonstrated ability to operate across product, engineering, research, and marketing organizations. Product Marketing IC4 - The typical base pay range for this role across the U.S. is USD $106,400 - $203,600 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $137,600 - $222,600 per year. Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: careers.microsoft.com This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled. Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.

Requirements

Must-haves

In alignment with our Microsoft values, we are committed to cultivating an inclusive work environment for all employees to positively impact our culture every day.
Build AI agents, applications, and automated workflows that accelerate product testing, competitive evaluation, validation, and readiness assessment.
Orchestrate the automation of end-to-end evaluation process—from test setup and execution through scoring, validation, analysis, and reporting.
Detect failures, regressions, inconsistencies, and anomalies at scale, with clear mechanisms for human review where judgment matters.
Prototype agentic evaluation approaches—including agents that test other agents—and rapidly move the strongest concepts into production workflows.
Turn evaluation signals into actionable product and competitive insights that shape prioritization, readiness, differentiation, and customer experience.
Scale AI automation across the organization through reusable tools, templates, documentation, responsible practices, and hands-on technical leadership.
Embody our Culture   & Values .
Required/minimum qualifications
Bachelor’s Degree AND 8+ years experience, OR Master’s Degree AND 6+ years experience
Experience building AI agents, agentic workflows, or prototypes using code, APIs, structured data, JSON, and developer tools.
Strong understanding of generative AI, LLMs, agents, and AI product experiences.
Experience with AI evaluation, quality measurement, experimentation, or automated testing, including concepts such as automated graders, human evaluation, judge models, benchmarks, and regression testing.
Strong analytical skills and the ability to design systematic approaches to testing and validation.
Excellent written and verbal communication skills, with the ability to make complex technical concepts clear to non-technical audiences.
Demonstrated ability to operate across product, engineering, research, and marketing organizations.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

Nice-to-haves

Additional or preferred qualifications

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

Platform: eightfold
Effort: medium