We launched recruiters.rip, let your AI apply for you. First month free with code onemonthfree

Engineering Manager - Inference Performance

Baseten is an AI inference platform for deploying, optimizing, and scaling custom, open-source, and fine-tuned models in production.

Series F+Recently funded0 current maintainers0 active leads
San Francisco, United States
About Baseten

Baseten provides model runtimes, inference infrastructure, developer workflows, and deployment options including managed cloud, self-hosted, and hybrid environments.

View jobs by Baseten

Skills

Candidate Availability

About the Role

You will provide hands-on technical leadership for inference performance work. You will manage, mentor, and hire engineers; set the technical roadmap; review designs; guide profiling and optimization; and improve GPU utilization, latency, cost, and runtime performance.

Requirements

  • Experience managing engineers, including hiring, mentoring, feedback, and performance reviews
  • Experience leading or supporting GPU optimization teams
  • Understanding of GPU architecture and performance tradeoffs
  • Familiarity with PyTorch, TensorRT, or TensorRT-LLM
  • Experience driving roadmaps and shipping complex technical projects
  • Written and verbal communication skills

Responsibilities

  • Lead, mentor, and grow inference performance engineers
  • Hire GPU and inference engineering talent
  • Own the technical roadmap and execution for runtime performance work
  • Review designs and guide profiling and optimization efforts
  • Drive productionization of quantization, speculative decoding, KV-cache reuse, chunked prefill, and custom scheduling
  • Measure tokens per GPU-hour, utilization, latency, and cost
  • Bring up and tune new model architectures on new hardware
  • Coordinate launches and priorities with partner functions
  • Set standards for engineering quality, benchmarking, operational excellence, and incident response

Benefits

  • Equity
  • Medical, dental, and vision insurance for U.S. employees and dependents
  • Flexible PTO
  • Company-wide Winter Break
  • Paid parental leave
  • Fertility and family-building stipend through Carrot
  • Company-facilitated 401(k) for U.S. employees