Member of Technical Staff - Inference
AI infrastructure company providing an integrated stack for training, evaluating, deploying, and continuously improving agentic models.
Maintainer signals as of 9/25/2026
Funding history
Projects
About Prime Intellect
Prime Intellect, Inc. operates AI infrastructure spanning RL environments, hosted training and evaluations, inference, secure sandboxes, and globally sourced GPU compute.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
Build infrastructure for efficient multi-tenant LLM serving across cloud GPU fleets, design GPU-aware scheduling and failover, optimize inference frameworks and parallelism, integrate distributed inference into RL systems, and establish CI/CD, observability, documentation, and incident response practices.
Requirements
- 3+ years building and operating large-scale ML or LLM services
- Experience with vLLM, SGLang, or TensorRT-LLM
- Familiarity with distributed and disaggregated serving infrastructure
- Understanding of prefill, decode, KV-cache behavior, batching, sampling, speculative decoding, and parallelism
- Experience debugging CUDA, NCCL, drivers, kernels, containers, service mesh, networking, and storage
- Python
- PyTorch
- AWS or GCP
- Kubernetes
- CUDA
- NCCL
- InfiniBand
- Nice-to-have experience with CUDA or Triton kernels, Nsight profiling, Rust, C++, Kafka or PubSub, Redis, gRPC or Protobuf, Prometheus or Grafana, OpenTelemetry, Terraform or Ansible, and open-source infrastructure contributions
Responsibilities
- Build a multi-tenant LLM serving platform across cloud GPU fleets
- Design GPU-aware placement and scheduling algorithms
- Implement multi-region and multi-zone failover and traffic shifting
- Build autoscaling, routing, and load balancing
- Optimize model distribution and cold-start times
- Integrate and contribute to LLM inference frameworks
- Optimize parallelism, caching, memory management, quantization, and speculative decoding
- Profile kernels, memory bandwidth, and transport
- Develop reproducible performance suites
- Embed distributed inference into the RL stack
- Establish CI/CD with artifact promotion and performance gates
- Build observability and manage SLOs
- Document architectures and playbooks
- Mentor and collaborate cross-functionally
Benefits
- Significant equity incentives
- Flexible work arrangement
- Full visa sponsorship
- Relocation support
- Professional development budget
- Regular team off-sites
- Conference attendance