Software Engineer, ML Workload Platform
Toronto-founded enterprise AI company building foundation models, agentic AI, enterprise search, and private deployment products.
About Cohere
Cohere provides enterprise-focused AI models and end-to-end products, including North for agentic workplace AI, Compass for enterprise search, and models for generation and retrieval. It supports private, VPC, and on-premises deployment options.
Skills
About the Role
You will own the platform that schedules and runs training, evaluation, and reinforcement-learning jobs on Kubernetes. You will build the networking and runtime isolation these workloads need, work with the data infrastructure team on checkpoint storage, troubleshoot production issues, and build automation and observability. The role includes a compensated 24x7 on-call rotation.
Requirements
- Experience running a job platform, batch system, or Kubernetes controllers in production
- Distributed systems judgment: retries, partial failure
- Experience with ML training workflows, RL sandboxes, container networking (CNI), or checkpoint storage at scale
- Software development in Python or Go
- Experience working with researchers or ML engineers
- Ability to find bottlenecks and drive fixes in a fast-moving environment
Responsibilities
- Own the platform that schedules and runs training, evaluation, and reinforcement-learning jobs on Kubernetes
- Build networking and runtime isolation for workloads, including agent and sandbox environments
- Work with the data infrastructure team on storage for checkpoints and training artifacts
- Troubleshoot production failures and performance problems
- Build automation, infrastructure-as-code, and observability
- Participate in a compensated 24x7 on-call rotation
Benefits
- Weekly lunch stipend of $75/£75
- Full health and dental benefits with separate mental health budget
- RRSP matching, 401K, Pension Scheme
- 100% parental leave top-up for up to 6 months
- Annual enrichment benefits
- Education and learning stipend
- 6 weeks of paid vacation
- Travel budget to other offices and annual company offsite
- Co-working benefit
- $500 home office stipend
- Equity