Staff / Principal Machine Learning Engineer, Serving
Inworld AI is a research lab and inference provider for realtime AI at consumer scale. It provides text-to-speech, speech-to-text, speech-to-speech, and LLM routing products for developers and businesses building consumer-facing applications.
Maintainer signals as of 9/25/2026
Funding history
Projects
About Inworld AI
Inworld AI develops and operates realtime AI infrastructure, including first-party text-to-speech and speech-to-text models, a customizable speech-to-speech Realtime API, and an LLM Router that routes requests across leading model providers. Its platform supports voice agents, social and companion applications, learning and education, health and wellness, agentic workforce applications, and games and media. The company serves developers, businesses, and teams building consumer-facing applications.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
Optimize sub-second multimodal inference for realtime voice models, taking models from research into production through containerization, serving optimization, and reliable operation. Apply model acceleration techniques and build high-performance distributed systems capable of handling thousands of concurrent connections.
Requirements
- Deep understanding of modern serving frameworks such as vLLM or TRT-LLM.
- Hands-on experience with model acceleration techniques.
- Proficiency in C++, CUDA, Rust, or highly optimized Python.
- Experience profiling code and optimizing NVIDIA GPU performance.
- Experience with Kubernetes, Ray, custom load balancing, and multi-GPU or multi-node inference.
- Experience handling thousands of concurrent connections reliably.
- Non-trivial systems programming projects, open-source contributions, or deep-dive technical write-ups.
- Ability to take models from research to production.
- PhD in CS, Physics, Math, or equivalent practical experience building backend or ML systems.
Responsibilities
- Optimize realtime inference serving with frameworks such as vLLM and TRT-LLM.
- Apply quantization, distillation, caching, continuous batching, paged attention, and speculative decoding.
- Build high-performance systems in C++, CUDA, Rust, or optimized Python and profile GPU performance.
- Design and scale distributed systems using Kubernetes, Ray, custom load balancing, and multi-GPU or multi-node inference.
- Handle thousands of concurrent connections reliably.
- Take models from research to production, including containerization and serving optimization.
- Contribute to open-source projects and publish technical write-ups.
Benefits
- Relocation assistance
- Bonus
- Equity
- Benefits