Member of Technical Staff, Robotics Research at Sieve

Job description

ABOUT US Sieve is a multi-modal lab curating the world's highest-quality training datasets — spanning video, audio, images, text, and 3D. We combine exabyte-scale data infrastructure and novel multimodal understanding techniques that push the frontier of foundation models. Video alone makes up 80% of internet traffic, and across modalities, data has become the enabling medium powering creativity, communication, gaming, AR/VR, and robotics. Sieve exists to solve the biggest bottleneck in the growth of these applications: high-quality training data. We partner with top AI labs and did $XXM last quarter alone, as a team of ~30 people. We also raised our Series A from Tier 1 firms such as Matrix Partners matrix.vc, Swift Ventures swift.vc, Y Combinator ycombinator.com, and AI Grant aigrant.com. ABOUT THE ROLE As our first Robotics Researcher at Sieve, you'll lead the training and evaluation of world-action models (WAMs) and 3D perception systems to understand how data shapes robot capabilities. Your work connects how we capture, curate, and label physical-world data to measurable gains in robot learning. You'll run your projects from question to answer: spot where a model falls short, decide what data might close the gap, set up the experiment, and test the result on tasks and environments the model hasn't seen. A lot of this work is in 3D. Raw multimodal and egocentric video becomes useful for robot learning only once it carries geometry, pose, and action, and you'll build the systems that add them. Simulation is a tool you'll use when it makes evaluation or augmentation cheaper, not the center of the job. WHAT YOU’LL DO - Data ablations for robot learning: Compare data sources such as egocentric human video, teleop, and robot rollouts, and vary quality, mix, embodiment, and supervision to learn which choices improve policies and world models. - World-action models: Train and post-train action-conditioned world models and policies, pushing them toward skills like following commanded actions, manipulating objects, and responding to camera motion. - 3D labeling from video: Build pipelines for depth, camera trajectories, hand/body/object pose, point tracks, and scene reconstruction, so raw footage becomes action-labeled data grounded in 3D. - Evaluation design: Lock the test tasks, environments, and embodiments before comparing models, and report clear, checkable conclusions about which data made the difference. - Simulation as a tool: Use simulators or learned world models for evaluation, augmentation, and sim-to-real checks. - From findings to datasets: Turn results into the curation filters, annotation specs, and capture protocols behind our pipelines and the data we deliver to partner labs. - Partner collaboration: Work with Sieve engineers and researchers at partner labs to pick the right problems and share what you find. - Publish what you learn: Share findings through papers, technical reports, blog posts, and open datasets or benchmarks where it makes sense, and help shape how the field thinks about data for robot learning. REQUIREMENTS - 4+ years of research or engineering experience in robot learning, embodied AI, or 3D computer vision. A PhD is welcome but not required; strong publications, open-source projects, or systems you can walk us through count just as much. - Fluent Python plus PyTorch or JAX: you can write, debug, and change training code yourself - Hands-on 3D vision work: multi-view geometry, depth, pose, tracking, or reconstruction - Have trained a policy or world model (VLA, diffusion policy, imitation learning, action-conditioned video model, etc.) and tested it on data it never saw in training - Rigorous about experiments: you set baselines, run ablations, and can tell whether a gain came from the data or from architecture, compute, or a leaky eval - Comfortable with large, messy datasets and diagnosing data quality issues - Strong communication skills and the ability to explain findings, tradeoffs, and uncertainty clearly - In-person at our SF HQ BONUS - Robot learning from egocentric or human video, including retargeting human motion onto robot embodiments - Have put a trained policy on real robot hardware and measured it in closed loop - Simulators like MuJoCo or Isaac Sim, used for evaluation or synthetic data - Large-scale distributed training - Experience as an early hire at a startup BENEFITS - 401(k) and full health insurance - Breakfast, lunch, and dinner covered, plus your choice of snacks - Ubers covered home *All roles at Sieve require you to be onsite in San Francisco 5 days per week

Requirements

Must-haves

World-action models: Train and post-train action-conditioned world models and policies, pushing them toward skills like following commanded actions, manipulating objects, and responding to camera motion.
3D labeling from video: Build pipelines for depth, camera trajectories, hand/body/object pose, point tracks, and scene reconstruction, so raw footage becomes action-labeled data grounded in 3D.
Evaluation design: Lock the test tasks, environments, and embodiments before comparing models, and report clear, checkable conclusions about which data made the difference.
Simulation as a tool: Use simulators or learned world models for evaluation, augmentation, and sim-to-real checks.
From findings to datasets: Turn results into the curation filters, annotation specs, and capture protocols behind our pipelines and the data we deliver to partner labs.
Partner collaboration: Work with Sieve engineers and researchers at partner labs to pick the right problems and share what you find.
Publish what you learn: Share findings through papers, technical reports, blog posts, and open datasets or benchmarks where it makes sense, and help shape how the field thinks about data for robot learning.
Fluent Python plus PyTorch or JAX: you can write, debug, and change training code yourself
Hands-on 3D vision work: multi-view geometry, depth, pose, tracking, or reconstruction
Have trained a policy or world model (VLA, diffusion policy, imitation learning, action-conditioned video model, etc.) and tested it on data it never saw in training
Rigorous about experiments: you set baselines, run ablations, and can tell whether a gain came from the data or from architecture, compute, or a leaky eval
Comfortable with large, messy datasets and diagnosing data quality issues
Strong communication skills and the ability to explain findings, tradeoffs, and uncertainty clearly
In-person at our SF HQ

Nice-to-haves

Robot learning from egocentric or human video, including retargeting human motion onto robot embodiments
Have put a trained policy on real robot hardware and measured it in closed loop
Simulators like MuJoCo or Isaac Sim, used for evaluation or synthetic data
Large-scale distributed training
Experience as an early hire at a startup

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

aiswiftpython

Platform: ashby
Effort: quick

Benefits

Benefits detected in the listing.

Health Insurance

Vision

401(K)