Research Engineer - Distributed Training

1 month agoSeniorSalary: 150K - 350KSan Francisco, USA Remote Full Time Ai Jobs by Prime Intellect

Skills

Candidate Availability

Required and preferred rules are kept separate and reflect the wording in the original posting.

About the Role

Build and optimize distributed training infrastructure for pre-training and large-scale reinforcement learning workloads, improving efficiency across compute, memory, networking, scheduling, and runtime systems.

Requirements

AI/ML infrastructure engineering experience
Large-scale model training or inference experience
PyTorch
PyTorch Distributed
DeepSpeed
FSDP
Megatron
vLLM
Ray
Training performance optimization
Data parallelism
Tensor parallelism
Pipeline parallelism
GPU architecture
Profiling
Performance debugging
CUDA
Triton
Compiler optimization
Runtime optimization
RL training infrastructure
Multi-node GPU clusters
High-performance networking
Open-source contributions

Responsibilities

Build and optimize distributed training infrastructure
Improve training efficiency across compute, memory, networking, and scheduling layers
Design and implement kernel, communication path, and runtime optimizations
Develop distributed training systems for data, tensor, and pipeline parallel workloads
Shape the architecture of the RL training stack
Contribute to open-source libraries and internal infrastructure
Translate system bottlenecks into concrete improvements
Track advances in training systems, inference systems, compiler/runtime tooling, and hardware-aware optimization

Benefits