Senior / Lead Machine Learning Engineer, Serving
Inworld AI is a research lab and inference provider for realtime AI at consumer scale. It provides text-to-speech, speech-to-text, speech-to-speech, and LLM routing products for developers and businesses building consumer-facing applications.
Maintainer signals as of 9/25/2026
Funding history
Projects
About Inworld AI
Inworld AI develops and operates realtime AI infrastructure, including first-party text-to-speech and speech-to-text models, a customizable speech-to-speech Realtime API, and an LLM Router that routes requests across leading model providers. Its platform supports voice agents, social and companion applications, learning and education, health and wellness, agentic workforce applications, and games and media. The company serves developers, businesses, and teams building consumer-facing applications.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
Work on optimizing realtime inference for state-of-the-art voice models, taking models from research into production by containerizing, optimizing, and operating them reliably. Build distributed systems for high-throughput inference and treat performance, latency, and reliability as first-class product features.
Requirements
- Deep understanding of modern serving frameworks and inference optimization techniques.
- Hands-on experience with quantization, distillation, caching, continuous batching, paged attention, and speculative decoding.
- Proficiency in C++, CUDA, Rust, or highly optimized Python.
- Experience with Kubernetes, Ray, custom load balancing, and multi-GPU or multi-node inference.
- Non-trivial systems programming projects, open-source contributions, or deep-dive technical write-ups.
- Full-cycle experience taking models from research to production.
- PhD in computer science, physics, or mathematics, or equivalent practical experience building backend or ML systems.
- Professional fluency in written and spoken English.
Responsibilities
- Optimize realtime inference using serving frameworks such as vLLM and TRT-LLM.
- Apply quantization, distillation, caching, continuous batching, paged attention, and speculative decoding.
- Profile and optimize NVIDIA GPU performance using C++, CUDA, Rust, or optimized Python.
- Build and scale distributed systems with Kubernetes, Ray, and custom load balancing.
- Support reliable multi-GPU and multi-node inference for thousands of concurrent connections.
- Take models from research through containerization, serving optimization, and reliable production operation.
- Design benchmarks and prototypes to validate architectural decisions.
- Collaborate daily with US-based leadership and engineering teams.
Benefits
- Full U.S. visa and relocation support may be available for candidates interested in relocating to the San Francisco Bay Area, subject to business needs and applicable legal and work authorization requirements.