Software Engineer - Inference Product
1 day agoSeniorSalary: 200000 - 400000 USD / yearSan Francisco, California, United States; New York, United StatesHybridFull TimeBackendJobs by Baseten
BasetenVisit Baseten website
Baseten is an AI inference platform for deploying, optimizing, and scaling custom, open-source, and fine-tuned models in production.
San Francisco, United States
Funding history
About Baseten
Baseten provides model runtimes, inference infrastructure, developer workflows, and deployment options including managed cloud, self-hosted, and hybrid environments.
Skills
Candidate Availability
About the Role
You'll build and operate the API layer that powers Model APIs, covering request handling, authentication, rate limiting and routing. You'll own usage metering and billing infrastructure, build capacity systems, ship new API products, add observability, and own projects end to end from design to on-call.
Requirements
- Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, or related field, or equivalent experience
- 3+ years building and operating distributed systems, backend infrastructure, or large-scale APIs
- Track record owning low-latency backend services including rate limiting, auth, quotas, metering, and migrations
- Performance skills: profiling, tracing, capacity planning, SLO management
- Ability to debug performance and reliability issues across the stack
- Strong sense of developer experience
- Eagerness to learn and interest in inference engineering
- Excellent written communication and collaboration skills
Responsibilities
- Build and operate the API layer behind Model APIs with high availability and low latency
- Own usage metering and billing infrastructure
- Build rate-limiting and capacity systems balancing throughput and fleet utilization
- Ship new products at the API layer
- Turn platform capabilities into a strong product experience
- Instrument the edge with observability and alerting
- Partner with Model APIs, inference and product teams
- Own projects end to end from design through deployment and on-call
Benefits
- Meaningful equity
- 100% coverage of medical, dental, and vision insurance for employee and dependents (U.S. only)
- Flexible PTO including company-wide Winter Break
- Paid parental leave
- Fertility and family-building stipend through Carrot
- Company-facilitated 401(k) (U.S. only)
