Performance Engineer Inference Engine
AI safety and research company building reliable, interpretable, and steerable AI systems, including the Claude product family and developer platform.
Funding history
About Anthropic
Anthropic PBC develops frontier AI systems and deploys them through Claude products and the Claude Platform, with a stated focus on safety, interpretability, and steerability.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will build and optimize inference-engine software across accelerator and cloud platforms. You will improve batching, model placement, memory management, forward-pass coordination, caching, observability, and performance modeling. You will also maintain model quality and support robust production safety systems while improving system efficiency.
Requirements
- Understanding of large language model inference, including prefill, decode, accelerator compute, memory, interconnect, and host behavior
- Systems programming in Rust, C++, or a similar language
- Performance profiling and hypothesis-driven optimization
- Experience learning unfamiliar systems and shipping consequential changes quickly
- Pair programming
- LLM serving engines
- GPU or accelerator programming
- Operating system internals
- Transformer language modeling
- Allocator, cache, scheduler, or high-bandwidth transport development
- Rust
- Determinism
- Replay
- Property-based testing
- Bachelor’s degree or equivalent education, training, or experience
Responsibilities
- Optimize inference throughput, cost, reliability, and latency across accelerator and cloud platforms
- Improve request batching, model placement, memory management, and forward-pass coordination
- Maintain and reuse model state when it is more efficient than recomputation
- Build observability for performance gaps and model potential improvements
- Deploy and measure performance improvements
- Maintain model quality across platforms and over time
- Support production safety systems without compromising robustness
Benefits
- Visa sponsorship support
- Equity donation matching
- Generous vacation
- Parental leave
- Flexible working hours