Software Engineer, Collective Communication

Skills

About the Role

You'll design and implement custom networking collectives tightly integrated into the training stack. You'll collaborate with ML researchers on efficient collective operations in C++ and CUDA, make sure the largest training jobs make full use of the network transports in the supercomputers, and work on simulations that inform future network designs.

Requirements

• Background in low level performance critical software
• Experience writing distributed algorithms using RDMA
• Comfortable writing low level performance-sensitive CPU and/or GPU code
• Familiarity with network simulation techniques

Responsibilities

• Collaborate with ML researchers to design and implement efficient collective operations in C++ and CUDA
• Ensure the largest training jobs take full advantage of the network transports in the supercomputers
• Work on simulations to inform future supercomputer network designs

Benefits

• Medical, dental, and vision insurance with employer HSA contributions
• Pre-tax Health FSA, Dependent Care FSA, and commuter accounts
• 401(k) with employer match
• Paid parental leave (up to 24 weeks birth parents, 20 weeks non-birthing parents) plus paid medical and caregiver leave (up to 8 weeks)
• Flexible PTO for exempt employees; up to 15 days for non-exempt
• 13+ paid company holidays and coordinated office closures, plus paid sick or safe time
• Mental health and wellness support
• Employer-paid basic life and disability coverage
• Annual learning and development stipend
• Daily meals in offices and meal delivery credits
• Relocation assistance
• Charitable donation matching and wellness stipends
• Equity
• Performance-related bonus