Job description

Overview Help build the world’s most advanced multimodal dataset at Microsoft AI We are on a mission to create the largest and most advanced multimodal dataset in the world. This dataset, spanning all modalities from across the web and beyond, will power the training of the world’s most capable AI frontier models, pushing the boundaries of scale, performance, and product deployment. The AI Data Infra team at Microsoft AI is responsible for building data infrastructure to help MAI teams to generate the biggest and best training dataset. Our work involves data pipelines, Spark, Ray, Vector Databases, and all other aspects of data infra. We are looking for outstanding individuals excited about contributing to the next generation of systems that will transform the field. In particular, we are looking for candidates who: Are passionate about the role of data in large-scale AI model training Will thrive in a highly collaborative, fast-paced environment Have a high degree of expertise and pay close attention to details Demonstrate a proactive attitude and enthusiasm for exploring new methods and technologies Effectively manage multiple responsibilities and can adjust to shifting priorities. Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond. Starting January 26, 2026, MAI employees are expected to work from a designated Microsoft office at least four days a week if they live within 50 miles (U.S.) or 25 miles (non-U.S., country-specific) of that location. This expectation is subject to local law and may vary by jurisdiction. This role is part of Microsoft AI's Superintelligence Team. The MAIST is a startup-like team inside Microsoft AI , created to push the boundaries of AI toward Humanist Superintelligence—ultra-capable systems that remain controllable, safety-aligned, and anchored to human values. Our mission is to create AI that amplifies human potential while ensuring humanity remains firmly in control. We aim to deliver breakthroughs that benefit society—advancing science, education, and global well-being. We’re also fortunate to partner with incredible product teams giving our models the chance to reach billions of users and create immense positive impact. If you’re a brilliant, highly-ambitious and low ego individual, you’ll fit right in—come and join us as we work on our next generation of models! Responsibilities Build AI Data Infrastructure and Data Engines Develop and evolve large-scale AI data infrastructure for frontier AI lab. Own data collection, ingestion, cleaning, curation, generation, governance, metadata management, query , analytic and hybrid search , building reusable, observable, and explainable EB-scale data platform that support high-quality data for pre-training and post-training workloads. Develop Intelligent Multimodal Data Processing Systems Lead automated understanding and processing of text, images, video, documents. Build labeling and taxonomy systems, semantic feature extraction, data quality modeling, and automated data governance capabilities. Design and train models for classification, recognition, captioning, quality scoring, prediction, and related data-processing tasks. Build AI-Native Data Pipelines Leverage LLMs, VLMs, and Agents to build intelligent data pipelines that automate data collection, filtering, deduplication, quality diagnosis, annotation/re-labeling, generation, scheduling, orchestration, and anomaly detection. Use AI-native workflows to significantly reduce manual data operations and improve pipeline scalability and efficiency. Build AI-Native Data Storage and Table Layers Design open, AI-optimized storage using Lance, Iceberg, Paimon, and Parquet. Support multimodal data and embeddings, fast random access and scans, schema evolution, transactions, versioning/ time travel, indexing, and interoperable access across training and query engines. Discover and Build Rare, High-Value Datasets Develop differentiated datasets for challenging domains, including web/PDF/encyclopedic/private/query-based text data as well as real-world multimodal scenarios such as retail inspection, warehouse inspection, healthcare, education, OCR, GUI interaction, and multimodal trajectories. Apply a combination of real-world data collection, human annotation, and synthetic data generation to produce rare and high-value datasets. Drive the Data–Model–Evaluation Iteration Loop Use evaluation feedback and model failure analysis to identify data gaps and design targeted datasets and synthetic-data strategies. Establish observable metrics that quantify the contribution of data to model capability improvements, continuously update datasets during training, and build an iterative Data → Model → Evaluation → Data optimization loop. Partner Closely with Model and Training Teams Collaborate with model researchers and training engineers on training-data construction, data feedback loops, targeted data mining, and evaluation-driven iteration. Use data as a primary lever for improving model capability and become a core driver of AI model training performance. Qualifications • Required: Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 3+ years experience in business analytics, data science, software development, data modeling, or data engineering OR Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 4+ years experience in business analytics, data science, software development, data modeling, or data engineering OR equivalent experience. Experience with distributed data Platforms such as Spark, Flink or Ray. Proficient experience in Python and experience with SQL and Shell. Experience with Multimodal Data (Text, Image, Video, or Audio) Preferred: • Hands-on experience building datasets for LLM/VLM/multimodal model pre-training or post-training . • Experience with synthetic data , including text generation, image-text synthesis, rendering/diffusion-based generation, or multimodal trajectory generation for GUI, search, file-operation, or agentic tasks. • Familiarity with major evaluation benchmarks such as MMLU, MMBench, MM-BrowseComp , with practical experience using evaluation results and failure analysis to drive targeted data improvements. • Experience with open file and table formats such as Lance, Iceberg, Paimon and Parquet,including schema evolution, versioning, transactions, indexing, and performance optimization for multimodal AI workloads. • Solid understanding of LLMs, speech/audio models, vision models, and multimodal models. Hands-on experience with large-scale AI data construction, cleaning, synthesis, or quality evaluation. • Experience building ETL systems, data models, data pipelines, or data warehouses is strongly preferred. Experience processing large-scale text, image, or video datasets is a plus. • Familiarity with Agents and modern LLM toolchains, with practical experience—or strong interest—in applying LLMs to data production, analysis, quality control, governance, and pipeline automation. Data Engineering IC5 - The typical base pay range for this role across the U.S. is USD $142,800 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year. Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: careers.microsoft.com Data Engineering IC6 - The typical base pay range for this role across the U.S. is USD $165,600 - $296,400 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $220,800 - $331,200 per year. Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: careers.microsoft.com This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled. Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.

Requirements

Must-haves

Help build the world’s most advanced multimodal dataset at Microsoft AI
We are looking for outstanding individuals excited about contributing to the next generation of systems that will transform the field. In particular, we are looking for candidates who:
Are passionate about the role of data in large-scale AI model training
Will thrive in a highly collaborative, fast-paced environment
Have a high degree of expertise and pay close attention to details
Demonstrate a proactive attitude and enthusiasm for exploring new methods and technologies
Effectively manage multiple responsibilities and can adjust to shifting priorities.
Build AI Data Infrastructure and Data Engines
Develop Intelligent Multimodal Data Processing Systems
Build AI-Native Data Pipelines
Build AI-Native Data Storage and Table Layers
Design open, AI-optimized storage using Lance, Iceberg, Paimon, and Parquet. Support multimodal
data and embeddings, fast random access and scans, schema evolution, transactions, versioning/
time travel, indexing, and interoperable access across training and query engines.
Discover and Build Rare, High-Value Datasets

Nice-to-haves

Hands-on experience building datasets for LLM/VLM/multimodal model pre-training or post-training .
Experience with synthetic data , including text generation, image-text synthesis, rendering/diffusion-based generation, or multimodal trajectory generation for GUI, search, file-operation, or agentic tasks.
Familiarity with major evaluation benchmarks such as MMLU, MMBench, MM-BrowseComp , with practical experience using evaluation results and failure analysis to drive targeted data improvements.
Experience with open file and table formats such as Lance, Iceberg, Paimon and Parquet,including schema evolution, versioning, transactions, indexing, and performance optimization for multimodal AI workloads.
Solid understanding of LLMs, speech/audio models, vision models, and multimodal models. Hands-on experience with large-scale AI data construction, cleaning, synthesis, or quality evaluation.
Experience building ETL systems, data models, data pipelines, or data warehouses is strongly preferred. Experience processing large-scale text, image, or video datasets is a plus.
Familiarity with Agents and modern LLM toolchains, with practical experience—or strong interest—in applying LLMs to data production, analysis, quality control, governance, and pipeline automation.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

aipythonsqlllm

Platform: eightfold
Effort: medium

Benefits

Benefits detected in the listing.

Vision