Sr. Software Engineer, Training/Inference (C++)
SpaceXAI builds frontier AI models and products including the Grok conversational assistant and a multimodal developer API.
Funding history
About SpaceXAI
SpaceXAI is a US-based AI company focused on accelerating scientific discovery and understanding the universe through reasoning, voice, image, video, and generative AI systems. Formerly xAI, it was acquired by SpaceX effective February 2, 2026, while continuing to operate the x.ai site and services.
Skills
Candidate Availability
About the Role
You will design and optimize large-scale model serving systems end-to-end, from distributed infrastructure such as global KV cache, load balancing, batch scheduling and auto-scaling to low-level optimizations including GPU kernels, quantization and speculative decoding. You will optimize latency and throughput under production workloads, build custom tracing and replay tools, create CI/CD infrastructure for inference deployment, and accelerate research on test-time compute scaling, RL rollout and model-hardware co-design.
Requirements
- Bachelor's degree in computer science, computer engineering, or science discipline
- 5+ years of low-level system programming experience in C/C++ or Rust
- Preferred: experience with large-scale high-concurrency production serving
- Preferred: experience with GPU inference engines (vLLM, SGLang, Triton, TensorRT-LLM)
- Preferred: background in batching, caching, load balancing, parallelism
- Preferred: GPU kernels and code generation
- Preferred: quantization, speculative decoding, distillation, low-precision numerics
- Preferred: testing, benchmarking, and reliability of inference services
- Preferred: CI/CD infrastructure for inference
Responsibilities
- Architect and implement scalable distributed infrastructure for model serving
- Optimize latency and throughput of model inference under production workloads
- Build and operate high-concurrency serving systems with excellent tail latency and reliability
- Benchmark, fine-tune, and accelerate inference engines, including low-level GPU kernel work and code generation
- Develop custom tools to trace, replay, and fix issues across the full stack
- Create robust CI/CD infrastructure for endpoint deployment, image publishing, and inference engine updates
- Accelerate research on scaling test-time compute, RL rollout, and model-hardware co-design
Benefits
- Equity
- Comprehensive medical, vision, and dental coverage
- 401(k) retirement plan
- Short & long-term disability insurance
- Life insurance
- Various discounts and perks