Job description

Overview The Outlook team at Microsoft is reimagining how people communicate, organize their work, and manage time through intelligent, trustworthy experiences across Mail and Calendar. We build products used at global scale, including Copilot-powered experiences for drafting, summarization, search, inbox prioritization, meeting preparation, scheduling, and agentic workflows. Our work spans product analytics, experimentation, telemetry, machine learning, generative AI, and large language model evaluation to improve customer value, quality, trust, and adoption across Outlook experiences. The Outlook Data Science team is looking for a Senior Data Scientist to lead high-visibility initiatives across Outlook and Copilot. In this role, you will partner with product, engineering, design, research, and applied science teams to define success, build trusted measurement systems, evaluate AI experiences, and translate complex data into product decisions. You will be expected to operate as a strategic thought partner, provide technical leadership, and help establish rigorous data science practices across the organization. Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond. Responsibilities • Lead the design and execution of analytical frameworks that evaluate product investments across Outlook Mail, Calendar, Search, and Copilot experiences. • Define metrics, value-creating interactions, guardrails, funnels, retention measures, and feature-level success criteria in partnership with product managers, engineers, and scientists. • Design and analyze online experiments, quasi-experiments, and observational studies to estimate causal impact and guide ship decisions. • Develop trusted dashboards, semantic models, and reporting pipelines that monitor product health, experimentation outcomes, AI quality, reliability, latency, sentiment, and adoption. • Evaluate generative AI and agentic experiences using offline and online evaluation methods, including benchmark datasets, human judgment, LLM-as-a-judge approaches, error analysis, and production telemetry. • Apply prompt engineering and evaluation techniques to improve response quality, grounding, relevance, safety, and task completion for Outlook scenarios. • Partner with applied scientists and engineers on model selection, feature development, post-training signals, and measurement strategies for AI-powered product experiences. • Use large-scale behavioral and telemetry data to identify customer needs, diagnose product issues, forecast trends, and uncover opportunities across commercial and consumer segments. • Influence product strategy by communicating clear, actionable insights to senior leaders and translating ambiguous business questions into measurable analytical plans. • Provide technical mentorship and raise the bar for statistical rigor, reproducibility, privacy-conscious analytics, and operational excellence across the data science community. Qualifications Required Qualifications: • Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 1+ year(s) data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results) • OR Master's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 3+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results) • OR Bachelor's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 5+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results) • OR equivalent experience. Other Requirements: Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter. Preferred Qualifications: • Experience with Generative AI technologies, such as large language models (LLMs), retrieval-augmented generation (RAG), agentic systems, and natural language processing (NLP), with an understanding of AI quality dimensions such as relevance, groundedness, factuality, safety, reliability, latency, and user trust. • Hands-on experience in AI evaluation and optimization, such as prompt engineering, benchmark and rubric design, LLM evaluation, human-in-the-loop assessment, A/B testing, offline evaluations, production telemetry analysis, and leveraging customer feedback to improve AI product performance. • Experience working with large, distributed datasets and building reusable analytical datasets, metrics layers, dashboards, or self-service analytics solutions. • Advanced proficiency with big-data technologies and cloud analytics platforms. Experience with Azure Data Explorer/Kusto, Azure Data Lake, Synapse, Databricks, Data Factory, or Power BI is a plus. • Experience driving product analytics for communication, productivity, search, collaboration, or enterprise software products, with a proven ability to lead ambiguous cross-functional initiatives, influence stakeholders, communicate insights effectively, and deliver measurable business impact at scale. • Advanced degree in a quantitative or technical field is preferred. • 6+ years of experience in data science, statistics, machine learning, product analytics, business intelligence, or data-driven product strategy. • 4+ years of experience using SQL and Python or R to conduct analysis and implement statistical or machine learning methods, such as regression, classification, clustering, forecasting, causal inference, or experimentation. • Experience defining product metrics and using large-scale data to influence roadmap, investment, and ship decisions. • Experience designing experiments and applying statistical methods to measure product or business impact. #Outlook; #OPG #OPGHiring Data Science IC4 - The typical base pay range for this role across the U.S. is USD $119,800 - $234,700 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $160,200 - $261,000 per year. Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: careers.microsoft.com This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled. Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.

Requirements

Must-haves

Lead the design and execution of analytical frameworks that evaluate product investments across Outlook Mail, Calendar, Search, and Copilot experiences.
Define metrics, value-creating interactions, guardrails, funnels, retention measures, and feature-level success criteria in partnership with product managers, engineers, and scientists.
Design and analyze online experiments, quasi-experiments, and observational studies to estimate causal impact and guide ship decisions.
Develop trusted dashboards, semantic models, and reporting pipelines that monitor product health, experimentation outcomes, AI quality, reliability, latency, sentiment, and adoption.
Evaluate generative AI and agentic experiences using offline and online evaluation methods, including benchmark datasets, human judgment, LLM-as-a-judge approaches, error analysis, and production telemetry.
Apply prompt engineering and evaluation techniques to improve response quality, grounding, relevance, safety, and task completion for Outlook scenarios.
Partner with applied scientists and engineers on model selection, feature development, post-training signals, and measurement strategies for AI-powered product experiences.
Use large-scale behavioral and telemetry data to identify customer needs, diagnose product issues, forecast trends, and uncover opportunities across commercial and consumer segments.
Influence product strategy by communicating clear, actionable insights to senior leaders and translating ambiguous business questions into measurable analytical plans.
Provide technical mentorship and raise the bar for statistical rigor, reproducibility, privacy-conscious analytics, and operational excellence across the data science community.
OR equivalent experience.
Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings:
Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
6+ years of experience in data science, statistics, machine learning, product analytics, business intelligence, or data-driven product strategy.
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

Nice-to-haves

Experience working with large, distributed datasets and building reusable analytical datasets, metrics layers, dashboards, or self-service analytics solutions.
Advanced proficiency with big-data technologies and cloud analytics platforms. Experience with Azure Data Explorer/Kusto, Azure Data Lake, Synapse, Databricks, Data Factory, or Power BI is a plus.
Advanced degree in a quantitative or technical field is preferred.
Experience defining product metrics and using large-scale data to influence roadmap, investment, and ship decisions.
Experience designing experiments and applying statistical methods to measure product or business impact.
#Outlook; #OPG #OPGHiring
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

machine learningaillmazuresqlpython

Platform: eightfold
Effort: medium