Product Manager, Inference Platform

Skills

About the Role

You will work alongside inference engineers and researchers to turn advances in batching, caching, quantization, speculative decoding, and serving architectures into product features. You'll define benchmarks, prioritize optimizations based on customer workloads, and partner on release readiness.

Requirements

• Experience building and shipping AI/ML products or infrastructure where performance, scalability, or production behavior mattered
• Technical background in software engineering, ML engineering, research, or equivalent
• Strong understanding of model serving architectures, GPU economics, and latency, throughput, quality, cost tradeoffs
• Experience working with researchers or infrastructure engineers to bring complex capabilities into production
• Rigorous approach to benchmarking, experimentation, and performance measurement
• Strong product judgment

Responsibilities

• Work with inference engineering and research teams to bring new performance capabilities to customers
• Translate advances in batching, caching, quantization, speculative decoding, and serving architectures into product features
• Define benchmarks and evaluations demonstrating improvements in latency, throughput, quality, and cost
• Understand customer workloads and prioritize high-impact optimizations
• Make performance features easier to configure, evaluate, and adopt in production
• Help customers understand tradeoffs across hardware, model configurations, autoscaling, and routing
• Partner with engineering on release readiness, compatibility, and regression testing

Benefits

• Equity
• 100% coverage of medical, dental, and vision insurance for employee and dependents (U.S. only)
• Flexible PTO policy including company-wide Winter Break
• Paid parental leave
• Fertility and family-building stipend through Carrot
• Company-facilitated 401(k) (U.S. only)