We launched recruiters.rip, let your AI apply for you. First month free with code onemonthfree

Software Engineer - Inference Product

Baseten is an AI inference platform for deploying, optimizing, and scaling custom, open-source, and fine-tuned models in production.

Series F+Recently funded0 current maintainers0 active leads
San Francisco, United States
About Baseten

Baseten provides model runtimes, inference infrastructure, developer workflows, and deployment options including managed cloud, self-hosted, and hybrid environments.

View jobs by Baseten

Skills

Candidate Availability

About the Role

You'll build and operate the API layer that powers Model APIs, covering request handling, authentication, rate limiting and routing. You'll own usage metering and billing infrastructure, build capacity systems, ship new API products, add observability, and own projects end to end from design to on-call.

Requirements

  • Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, or related field, or equivalent experience
  • 3+ years building and operating distributed systems, backend infrastructure, or large-scale APIs
  • Track record owning low-latency backend services including rate limiting, auth, quotas, metering, and migrations
  • Performance skills: profiling, tracing, capacity planning, SLO management
  • Ability to debug performance and reliability issues across the stack
  • Strong sense of developer experience
  • Eagerness to learn and interest in inference engineering
  • Excellent written communication and collaboration skills

Responsibilities

  • Build and operate the API layer behind Model APIs with high availability and low latency
  • Own usage metering and billing infrastructure
  • Build rate-limiting and capacity systems balancing throughput and fleet utilization
  • Ship new products at the API layer
  • Turn platform capabilities into a strong product experience
  • Instrument the edge with observability and alerting
  • Partner with Model APIs, inference and product teams
  • Own projects end to end from design through deployment and on-call

Benefits

  • Meaningful equity
  • 100% coverage of medical, dental, and vision insurance for employee and dependents (U.S. only)
  • Flexible PTO including company-wide Winter Break
  • Paid parental leave
  • Fertility and family-building stipend through Carrot
  • Company-facilitated 401(k) (U.S. only)