Senior Machine Learning Engineer Inference
Inworld AI is a research lab and inference provider for realtime AI at consumer scale. It provides text-to-speech, speech-to-text, speech-to-speech, and LLM routing products for developers and businesses building consumer-facing applications.
Funding history
Projects
About Inworld AI
Inworld AI develops and operates realtime AI infrastructure, including first-party text-to-speech and speech-to-text models, a customizable speech-to-speech Realtime API, and an LLM Router that routes requests across leading model providers. Its platform supports voice agents, social and companion applications, learning and education, health and wellness, agentic workforce applications, and games and media. The company serves developers, businesses, and teams building consumer-facing applications.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will optimize and serve machine learning models for production inference. You will profile and accelerate GPU workloads, improve latency and throughput, containerize models, scale multi-GPU and multi-node serving, and ensure reliable operation in production.
Requirements
- Deep understanding of modern serving frameworks and inference optimization techniques such as vLLM or TRT-LLM
- Hands-on experience with quantization, distillation, caching strategies, continuous batching, paged attention, and speculative decoding
- Proficiency in C++, CUDA, Rust, or highly optimized Python
- Ability to profile code and optimize NVIDIA GPU performance
- Experience with Kubernetes, Ray, load balancing, and distributed inference systems
- Experience handling thousands of concurrent connections
- Evidence of non-trivial systems programming projects, open-source contributions, or technical write-ups
- Ability to own the lifecycle from research model through production serving
- PhD in Computer Science, Physics, or Mathematics, or equivalent practical experience building backend or ML systems
Responsibilities
- Optimize model-serving and inference performance
- Apply quantization, distillation, caching, continuous batching, paged attention, and speculative decoding
- Profile and accelerate workloads on NVIDIA GPUs
- Build and scale multi-GPU and multi-node inference systems
- Containerize models and optimize their production serving
- Ensure reliable production operation for high-concurrency inference
Benefits
- Relocation assistance
- Bonus
- Equity
- Benefits