Senior Software Engineer ML API AI Inference
Cerebras builds wafer-scale AI computing systems and a cloud inference platform for training, fine-tuning, and serving AI models.
About Cerebras Systems, Inc.
Cerebras Systems is an AI-infrastructure company founded in 2015. It sells rack-scale wafer-scale computing systems and provides cloud-based, API-accessible AI inference alongside on-premises deployments.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will design, implement, test, and operate ML API features for a disaggregated AI inference platform. You will integrate models and serving components, improve routing, performance, reliability, and release quality, and lead technical delivery across multiple components. You will also mentor engineers and improve developer and customer workflows.
Requirements
- 5+ years of software engineering experience
- Strong Python programming ability
- Proficiency in Go, C++, Rust, or another production-services language
- Experience building production APIs
- Knowledge of REST, gRPC, or streaming protocols
- Experience with Linux, containers, Kubernetes or comparable orchestration, and CI/CD
- Experience diagnosing distributed-system correctness, reliability, and performance issues
- Bachelor's degree in a related discipline or equivalent practical experience
Responsibilities
- Design, implement, test, and operate features for chat completions, text generation, streaming, model configuration, tool calling, structured outputs, and multimodal inputs
- Write design documents, evaluate tradeoffs, plan milestones, and coordinate multi-component delivery
- Implement consistent request and response behavior across inference backends
- Integrate foundation models, tokenizers, chat templates, generation configuration, sampling methods, and model-specific features
- Extend inference services with vLLM, PyTorch, Hugging Face libraries, LLGuidance, and Cerebras runtime components
- Implement routing, state transfer, error handling, retries, cancellation, and request lifecycle management
- Diagnose bottlenecks and improve latency, throughput, and resource efficiency
- Build conformance, validation, integration, and performance-regression tests
- Improve metrics, tracing, logging, alerts, diagnostic tools, resilience, and incident response
- Improve configuration, documentation, examples, and debugging workflows
- Review designs and code, mentor engineers, and remove delivery roadblocks