Software Engineer - Model Products
52 minutes agoSeniorSalary: 180K - 360KSan Francisco; Montreal; New York; Seattle; TorontoHybridFull TimeBackendJobs by Baseten
BasetenVisit Baseten website
Baseten is an AI inference platform for deploying, optimizing, and scaling custom, open-source, and fine-tuned models in production.
San Francisco, United States
Funding history
About Baseten
Baseten provides model runtimes, inference infrastructure, developer workflows, and deployment options including managed cloud, self-hosted, and hybrid environments.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will build and operate model APIs for advanced inference, including structured outputs, tool calling, and multimodal serving. You will optimize runtimes and multi-GPU performance, create benchmarks and observability, and implement platform capabilities such as versioning, validation, metering, quotas, and authentication.
Requirements
- Distributed system
- API
- Backend service
- Rate limiting
- Authentication
- Quota
- Usage metering
- Migration
- Profiling
- Tracing
- Capacity planning
- SLO management
- Design documentation
- LLM runtime
- vLLM
- SGLang
- TensorRT-LLM
- TGI
- Kubernetes
- Service mesh
- API gateway
- Distributed scheduling
- CUDA
- TensorRT
- Quantization
- Batching
- KV cache
Responsibilities
- Design, build, and operate Model APIs
- Implement structured outputs, tool and function calling, and multimodal serving
- Profile and optimize TensorRT-LLM and CUDA kernel performance
- Implement custom CUDA operators and optimize multi-GPU communication
- Productionize runtime improvements including speculative decoding, quantization, batching, and KV-cache reuse
- Build benchmarking frameworks
- Instrument metrics, traces, and logs
- Implement API versioning, validation, usage metering, quotas, and authentication
- Collaborate to deliver developer-friendly model serving
Benefits
- Equity
- Medical insurance for employees and dependents in the U.S.
- Dental insurance for employees and dependents in the U.S.
- Vision insurance for employees and dependents in the U.S.
- Flexible PTO
- Company-wide Winter Break
- Paid parental leave
- Fertility and family-building stipend through Carrot
- Company-facilitated 401(k) in the U.S.