Engineering Manager GPU Infrastructure

Toronto-founded enterprise AI company building foundation models, agentic AI, enterprise search, and private deployment products.

Toronto, Canada
About Cohere

Cohere provides enterprise-focused AI models and end-to-end products, including North for agentic workplace AI, Compass for enterprise search, and models for generation and retrieval. It supports private, VPC, and on-premises deployment options.

View jobs by Cohere

Skills

About the Role

You will lead and develop GPU infrastructure engineers, set the fleet technical roadmap, and scale Kubernetes clusters. You will partner with researchers and stakeholders, improve reliability and observability, automate cluster provisioning, optimize costs, and manage vendor relationships.

Requirements

  • Experience managing engineering or SRE teams
  • Experience with technical mentorship, hiring, and career development
  • Experience operating large Kubernetes compute fleets in production
  • Experience with multi-cloud infrastructure, multi-cluster operations, scheduling, node health, infrastructure as code, and monitoring
  • Knowledge of cluster operations, GPU networking, or hardware
  • Experience with GPU infrastructure cost optimization and capacity planning
  • Experience partnering with researchers or ML engineers
  • Ability to make data-informed tradeoffs across reliability, cost, and delivery

Responsibilities

  • Hire, mentor, and develop GPU infrastructure engineers
  • Own the technical roadmap for deploying, operating, and scaling Kubernetes clusters
  • Partner with researchers and ML engineers on training and inference infrastructure
  • Coordinate planning, cost, compliance, and dependencies with cross-functional stakeholders
  • Improve GPU utilization and reliability through observability, automation, and cost optimization
  • Manage vendor relationships

Benefits

  • Equity
  • Weekly lunch stipend
  • Health and dental benefits
  • Mental health budget
  • RRSP matching
  • 401K
  • Pension Scheme
  • Parental leave top-up for up to six months
  • Arts and culture benefit
  • Fitness and wellness benefit
  • Workspace improvement credit
  • Six weeks of paid vacation
  • Travel budget for other offices
  • Annual company offsite
  • Co-working benefit
  • Home office stipend