Job description

As a Research Engineer on the Pretraining Data team, you will own technically challenging projects spanning research, algorithm development, experimentation, and production engineering. You will investigate new approaches to improving training data quality, diversity, and efficiency, while building robust and scalable infrastructure to support the processing of massive datasets. You will collaborate closely with model training teams to understand the relationship between data and model capabilities, design controlled experiments, and continuously improve pretraining strategies. BS, MS, or PhD in Computer Science, Machine Learning, or a related technical field, or equivalent practical experience. Strong programming skills in Python and proficiency in developing production-quality software. Experience with large-scale data processing, distributed computing, or machine learning systems. Solid understanding of machine learning fundamentals and modern deep learning architectures. Strong analytical and problem-solving skills, with the ability to independently investigate complex technical challenges. Experience with LLM pretraining, training data research, or foundation model development. Experience designing and implementing scalable data processing pipelines using Spark, Ray, or similar distributed computing frameworks. Hands-on experience with PyTorch, model training, and experimental evaluation. Familiarity with data filtering, deduplication, data mixture optimization, and learned data quality signals. Understanding of scaling laws, training dynamics, and the relationship between data and model performance. Experience with synthetic data generation, data selection, or model-based data evaluation. Demonstrated ability to translate research ideas into scalable production systems. Publications or meaningful contributions to machine learning research or open-source projects.

Requirements

Must-haves

As a Research Engineer on the Pretraining Data team, you will own technically challenging projects spanning research, algorithm development, experimentation, and production engineering.
BS, MS, or PhD in Computer Science, Machine Learning, or a related technical field, or equivalent practical experience.
Strong programming skills in Python and proficiency in developing production-quality software.
Experience with large-scale data processing, distributed computing, or machine learning systems.
Solid understanding of machine learning fundamentals and modern deep learning architectures.
Strong analytical and problem-solving skills, with the ability to independently investigate complex technical challenges.
Experience with LLM pretraining, training data research, or foundation model development.
Experience designing and implementing scalable data processing pipelines using Spark, Ray, or similar distributed computing frameworks.
Hands-on experience with PyTorch, model training, and experimental evaluation.
Familiarity with data filtering, deduplication, data mixture optimization, and learned data quality signals.
Understanding of scaling laws, training dynamics, and the relationship between data and model performance.
Experience with synthetic data generation, data selection, or model-based data evaluation.
Demonstrated ability to translate research ideas into scalable production systems.
Publications or meaningful contributions to machine learning research or open-source projects.

Nice-to-haves

None flagged.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

machine learningpythonllm

Platform: apple
Effort: medium