Staff Software Engineer Inference Infrastructure
45 minutes agoLeadSalary: 190K - 325KSan Francisco; Montreal; New York; TorontoHybridFull TimeEngineeringJobs by Cohere
CohereVisit Cohere website
Toronto-founded enterprise AI company building foundation models, agentic AI, enterprise search, and private deployment products.
Toronto, Canada
About Cohere
Cohere provides enterprise-focused AI models and end-to-end products, including North for agentic workplace AI, Compass for enterprise search, and models for generation and retrieval. It supports private, VPC, and on-premises deployment options.
Skills
About the Role
You will work with multiple teams to deploy optimized NLP models into low-latency, high-throughput, highly available production environments. You will also interface with customers and create customized deployments that meet their needs.
Requirements
- 5+ years of engineering experience operating large-scale production infrastructure
- Highly available distributed systems, Kubernetes, and GPU workloads
- Kubernetes development, production coding, and support
- GCP, Azure, AWS, OCI, and multi-cloud or hybrid serving
- Linux computing environments
- Compute, storage, network resource, and cost management
- Troubleshooting
- GPU, TPU, or custom accelerator knowledge
- Distributed systems
- Golang or C++
Responsibilities
- Deploy optimized NLP models to production
- Build low-latency, high-throughput, highly available model-serving environments
- Create customized deployments for customers
Benefits
- Weekly lunch stipend of $75/£75 or local equivalent
- Health and dental benefits, including a mental-health budget
- RRSP matching, 401K, and pension scheme
- Parental leave top-up for up to 6 months
- Arts, culture, fitness, wellness, quality-time, and workspace-improvement benefits
- Education and learning stipend
- 6 weeks of paid vacation
- Travel budget for other offices and an annual company offsite
- Co-working benefit
- $500 home-office stipend