Lead Inference Engineer

Skills

About the Role

You will own the technical direction of inference serving and performance measurement. You will optimize GPU and distributed inference systems, develop benchmarking and instrumentation, improve latency and cost, define technical roadmaps and standards, and guide engineers while remaining hands-on with implementation.

Requirements

• Hands-on experience building, optimizing, or operating production LLM inference-serving systems
• Deep understanding of vLLM, SGLang, TensorRT-LLM, or similar inference stacks
• Understanding of paged attention, continuous batching, disaggregated serving, KV-cache management, and modern inference architectures
• GPU performance engineering experience using CUDA or Triton
• Understanding of GPU memory bandwidth, compute utilization, model bandwidth utilization, FLOPs utilization, FP8, and FP4
• Experience serving Mixture-of-Experts models including expert parallelism, all-to-all communication, and expert-load balancing
• Experience with model quantization and quality measurement against full-precision references
• Benchmarking experience including load generation, latency percentiles, throughput-at-SLO, quality measurement, and reproducibility
• Understanding of distributed systems routing and scheduling across varied hardware
• Ability to analyze performance bottlenecks across models, infrastructure, networking, and hardware
• Technical leadership experience in inference, serving, ML infrastructure, or complex engineering initiatives
• Advanced English

Responsibilities

• Own and improve the inference serving stack
• Improve throughput, latency, memory efficiency, and serving cost while maintaining model-quality standards
• Identify and eliminate GPU performance bottlenecks using CUDA, Triton, profiling, kernel optimization, memory optimization, and precision strategies
• Drive efficient serving of Mixture-of-Experts models
• Implement and evaluate quantization and model-optimization techniques
• Own reproducible benchmarking across performance, quality, and cost
• Build instrumentation to understand system behavior and validate improvements
• Make decisions on routing, scheduling, caching, and workload distribution
• Define the AI serving and measurement roadmap
• Establish standards for inference performance, benchmarking, reproducibility, testing, and production readiness
• Guide and mentor engineers across serving, metrics, and testing

Benefits

• Performance-based incentives
• 24 days annual leave plus public holidays
• Health insurance