Research Scientist - Pre-training
Pluralis Research is a research lab developing decentralized AI through Protocol Learning, a communication-efficient approach to collaborative model training. It operates open training systems that allow distributed contributors to provide compute for collectively owned foundation models.
Maintainer signals as of 9/25/2026
Funding history
About Pluralis Research
Pluralis Research develops Protocol Learning, which enables foundation models to be trained and served across globally distributed participants without requiring a single participant to hold the complete model. Its work covers low-bandwidth model parallelism, asynchronous distributed optimization, fault-tolerant training, privacy-preserving unextractable models, and collective ownership. The organization operates Agora and Node0 training systems, publishes research, provides participation documentation, and releases open-source software for distributed training and reinforcement-learning workflows.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will build the distributed training system for frontier-scale models across heterogeneous hardware and regions connected by ordinary internet. You will implement and optimize data, pipeline, and tensor parallelism, reduce communication overhead while preserving convergence, and improve elasticity and fault tolerance. You will also build monitoring for throughput, bottlenecks, and model quality across hundreds of devices.
Requirements
- Have hands-on distributed-training experience with PyTorch, FSDP, DeepSpeed, Megatron, or an equivalent implementation
- Understand data, tensor, and pipeline parallelism
- Have production-quality Python engineering skills, including concurrency, failure handling, and profiling
- Demonstrate execution through shipped systems, research code, open-source work, or serious personal projects
- Have professional-level written and spoken English proficiency
- Be comfortable working across time zones
Responsibilities
- Implement and optimize data, pipeline, and tensor-parallel training for large models on heterogeneous GPUs
- Implement techniques that reduce communication overhead while maintaining model convergence
- Build fault-tolerant checkpointing, state synchronization, and recovery for node churn
- Build monitoring for throughput, bottlenecks, and model quality across hundreds of devices
Benefits
- Significant equity in addition to a high base salary
- Flexible remote-first work environment
- Optional full visa sponsorship and relocation support to Australia or the US