Research Engineer Pre Training
Pluralis Research is a research lab developing decentralized AI through Protocol Learning, a communication-efficient approach to collaborative model training. It operates open training systems that allow distributed contributors to provide compute for collectively owned foundation models.
Funding history
About Pluralis Research
Pluralis Research develops Protocol Learning, which enables foundation models to be trained and served across globally distributed participants without requiring a single participant to hold the complete model. Its work covers low-bandwidth model parallelism, asynchronous distributed optimization, fault-tolerant training, privacy-preserving unextractable models, and collective ownership. The organization operates Agora and Node0 training systems, publishes research, provides participation documentation, and releases open-source software for distributed training and reinforcement-learning workflows.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will implement and optimize model-parallel training across heterogeneous GPUs and challenging networks. You will reduce communication overhead, improve fault tolerance and recovery, and build monitoring for throughput, bottlenecks, and model quality across distributed training runs.
Requirements
- Hands-on distributed training experience in PyTorch
- Experience with FSDP, DeepSpeed, Megatron, or equivalent implementations
- Understanding of data, tensor, and pipeline parallelism
- Production-quality Python skills
- Experience with concurrency, failure handling, and profiling
- Evidence of shipped systems, research code, open-source work, or serious projects
- Professional-level written and spoken English
Responsibilities
- Implement and optimize model-parallel training
- Reduce communication overhead while maintaining model convergence
- Build fault-tolerant checkpointing, synchronization, and recovery
- Build monitoring for throughput, bottlenecks, and model quality
Benefits
- Significant ownership for key technical contributors
- Flexible remote work environment
- Optional visa sponsorship and relocation support to Australia or the US