Member of Technical Staff, LLM & Production at Hark

Job description

About Hark Hark is an artificial intelligence company building advanced, personalized intelligence. One that is proactive, multimodal, and capable of interacting with the world through speech, text, vision, and persistent memory. We're pairing that intelligence with next-generation hardware to create a universal interface between humans and machines. While today's AI largely operates through chat boxes and decade-old devices, Hark is focused on what comes next: agentic systems that interact naturally with people and the real world. To get there, we're developing multimodal models and next-generation AI hardware together - designed from the ground up as a single, unified interface for a new era of intelligent systems. About the Role The Omni team at Hark is building the next generation of AI experiences beyond text, enabling models to understand and generate content across multiple modalities, including text, audio, and vision. Our goal is to create seamless, real-time multimodal intelligence that powers intuitive and immersive user experiences. We're looking for an engineer who sits at the intersection of model research and production systems. Together with our researchers, you'll decide what "better" means for a model and how to prove it — then build the infrastructure that makes those proofs reliable and repeatable: evaluation harnesses, data pipelines, and model training that hold up at scale and that the whole team can build on. This is a hands-on, individual-contributor position. The systems that judge whether a model improved, and the environments that drive that improvement, will be yours. Responsibilities Build and own end-to-end evaluation suites for LLM capabilities: task specs, grading logic, and the dashboards researchers check every day. Run the eval harness as a production service — sandboxed execution, tool calling, multi-turn and agentic tasks, deterministic replay, and caching — so a full evaluation pass is cheap and gives the same answer twice. Own the data pipeline from source to training: collection, filtering, dedup, labeling workflows, synthetic data generation, and the quality gates that catch contamination and distribution drift before anything reaches a run. Build RL training environments — reward functions, simulators, verifiers, task generators — and keep them stable as they fan out across many workers. Work side by side with researchers: turn a hypothesis into a runnable eval or environment within days, then help make sense of what comes back. Pull capabilities out of the product layer and into the model. Find where product teams lean on long prompt instructions or hand-written skills, build the evals and environments that teach the model to do those things natively, and retire the workarounds once it can. Measure everything: eval regressions, environment health, throughput, cost — and make it obvious when a metric moved for the wrong reason. Raise the bar for research code across the team: tests, versioned datasets and evals, and clean boundaries between training, inference, and evaluation. Requirements 3+ years in software engineering, including 1-2 years hands-on with LLM training, evaluation, or data infrastructure. Deep Python fluency and real distributed-systems experience: you've chased failures across a fleet of machines, dealt with unreliable workers, and know where the compute budget actually goes. You've built evaluation infrastructure, not just run benchmarks — designing tasks, graders, and the plumbing around them. Working knowledge of SFT and RL for language models. Production instincts: you write tests, insist on reproducibility, and treat research tooling as something others depend on — because they do. Comfortable in ambiguity next to researchers: handed a half-formed idea, you can make it measurable. Bonus Qualifications Agentic evaluation experience: tool use, code execution, sandbox environments, multi-turn tasks. Hands-on with training frameworks and inference stacks (e.g., PyTorch, JAX, vLLM, SGLang, or in-house equivalents). Experience running data-labeling operations or collecting human preferences at scale. Contributions to open-source evaluation frameworks or public benchmarks. Previous work on verifiers, code sandboxes, or simulation environments. Compensation The US base salary range for this full-time position is between $180,000 - $450,000 annually. The pay offered for this position may vary based on several individual factors, including job-related knowledge, skills, and experience. The total compensation package may also include additional components/benefits depending on the specific role. This information will be shared if an employment offer is extended.

Requirements

Must-haves

This is a hands-on, individual-contributor position. The systems that judge whether a model improved, and the environments that drive that improvement, will be yours.
Build and own end-to-end evaluation suites for LLM capabilities: task specs, grading logic, and the dashboards researchers check every day.
Run the eval harness as a production service — sandboxed execution, tool calling, multi-turn and agentic tasks, deterministic replay, and caching — so a full evaluation pass is cheap and gives the same answer twice.
Build RL training environments — reward functions, simulators, verifiers, task generators — and keep them stable as they fan out across many workers.
Work side by side with researchers: turn a hypothesis into a runnable eval or environment within days, then help make sense of what comes back.
Measure everything: eval regressions, environment health, throughput, cost — and make it obvious when a metric moved for the wrong reason.
Raise the bar for research code across the team: tests, versioned datasets and evals, and clean boundaries between training, inference, and evaluation.
3+ years in software engineering, including 1-2 years hands-on with LLM training, evaluation, or data infrastructure.
Deep Python fluency and real distributed-systems experience: you've chased failures across a fleet of machines, dealt with unreliable workers, and know where the compute budget actually goes.
You've built evaluation infrastructure, not just run benchmarks — designing tasks, graders, and the plumbing around them.
Working knowledge of SFT and RL for language models.
Production instincts: you write tests, insist on reproducibility, and treat research tooling as something others depend on — because they do.
Comfortable in ambiguity next to researchers: handed a half-formed idea, you can make it measurable.
Agentic evaluation experience: tool use, code execution, sandbox environments, multi-turn tasks.
Hands-on with training frameworks and inference stacks (e.g., PyTorch, JAX, vLLM, SGLang, or in-house equivalents).
Experience running data-labeling operations or collecting human preferences at scale.
Contributions to open-source evaluation frameworks or public benchmarks.
Previous work on verifiers, code sandboxes, or simulation environments.

Nice-to-haves

Bonus Qualifications

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

aillmpython

Platform: greenhouse
Effort: quick

Benefits

Benefits detected in the listing.

Vision