Research Engineer Pretraining

12 hours agoSalary: 150K - 350KSan Francisco, USA Remote Full Time Ai Jobs by Prime Intellect

Skills

Ablation Attention Checkpointing Distributed Training Language Model Midtraining Model Architecture Moe Numerical Stability Optimization Pretraining Python Pytorch

Candidate Availability

Remote · San Francisco · Required

Required and preferred rules are kept separate and reflect the wording in the original posting.

About the Role

You will extend pretraining code, move experiments into full training runs, evaluate model architectures, optimize training throughput, improve run resilience, investigate numerical stability, run reproducible ablations, and collaborate on training mixtures and long-context experiments.

Requirements

Experience training language models
Experience modifying training code and evaluating actual runs
Python skills
Experience with PyTorch or a comparable framework
Ability to debug distributed training
Understanding of model architectures and optimization
Ability to design and interpret ablations

Responsibilities

Build and extend pretraining code
Take small experiments into full training runs
Evaluate model architecture changes against quality and training cost
Profile training jobs and improve end-to-end throughput
Make long runs resilient through checkpointing and recovery
Investigate training dynamics and numerical stability
Run controlled and reproducible ablations
Work on training mixtures, midtraining, and long-context experiments

Benefits