Forward Deployed Engineer Inference
Toronto-founded enterprise AI company building foundation models, agentic AI, enterprise search, and private deployment products.
About Cohere
Cohere provides enterprise-focused AI models and end-to-end products, including North for agentic workplace AI, Compass for enterprise search, and models for generation and retrieval. It supports private, VPC, and on-premises deployment options.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will work directly with customer engineering teams to design and deploy reliable, secure, high-performance LLM inference systems. You will architect model-serving infrastructure, optimize deployment performance, validate hardware, investigate production issues, and translate field feedback into platform improvements.
Requirements
- 8+ years of engineering experience running production infrastructure, designing large-scale distributed systems, or deploying enterprise software
- Deep experience with production Kubernetes across multi-cluster and hybrid or multi-cloud ecosystems
- Understanding of LLM architecture and inference dynamics, including attention, prefill, and decode phases
- Experience with inference engines and optimization stacks such as vLLM, TensorRT-LLM, lmd, or Dynamo
- Familiarity with disaggregated serving, distributed KV caching, and multi-GPU workloads
- Customer-facing technical leadership and experience scoping ambiguous problems with executive stakeholders
- Programming foundations in Python, Go, or C++
Responsibilities
- Embed with customer engineering teams to connect AI models to complex production environments
- Design multi-tenant model-serving topologies, routing, and cross-datacenter failover strategies
- Optimize private and AI-in-a-box deployments across Kubernetes multi-cluster environments
- Benchmark and profile LLM workloads using metrics, dashboards, and distributed logs
- Validate and support model deployments on GPUs, TPUs, AMD hardware, and custom ASICs
- Translate field issues into platform upgrades and product-feature feedback
- Establish deployment patterns, SLOs, and observability frameworks for production reliability
Benefits
- Weekly lunch stipend of $75/£75 or local-currency equivalent
- Health and dental benefits, including a mental-health budget
- RRSP matching, 401K, and Pension Scheme
- 100% parental-leave top-up for up to 6 months
- Annual arts and culture, fitness and wellness, quality-time, workspace-improvement, and education stipends
- 6 weeks of paid vacation
- Travel budget for other offices and an annual company offsite
- Daily lunch, snacks, and social events for office-based employees
- Co-working benefit for employees not near an office
- $500 home-office stipend