Research Engineer Infrastructure

1 hour agoSalary: 230K - 450KSan Francisco, USA Full Time Ai Jobs by Harvey

Skills

Backend Data System Distributed System Distributed Training Evaluation Gpu Inference Jax Machine Learning Model Serving Observability Python Pytorch Reinforcement Learning

Candidate Availability

San Francisco · Required

Required and preferred rules are kept separate and reflect the wording in the original posting.

About the Role

You will build and scale infrastructure for post-training open-weight models, including distributed training, RL rollouts, reproducible agent environments, evaluation systems, and internal research tools. You will improve training performance and reliability, remove bottlenecks, integrate models into production, and work with researchers to turn experiments into dependable systems.

Requirements

Experience building ML infrastructure, distributed systems, or research tooling.
Python and generalist software engineering skills across backend services, data systems, and internal tools.
Familiarity with PyTorch, JAX, or similar ML frameworks.
Experience with distributed training or inference.
Ability to diagnose performance bottlenecks and failures across the stack.
Practical judgment about building versus adopting systems.

Responsibilities

Scale the post-training stack for SFT and RL.
Build distributed rollout infrastructure for model inference, tool execution, and training-experience collection.
Build reproducible agent environments for training and evaluation.
Own training-run performance and reliability, including GPU utilization, checkpointing, recovery, and observability.
Identify and remove training and inference bottlenecks.
Build tools for training-data management, evaluation, experiments, and training jobs.
Integrate models into production with serving, latency, cost, and online evaluation considerations.
Turn experimental workflows into dependable systems with researchers.

Benefits

Equity