Engineering Manager, Kubernetes Bare Metal Infrastructure
Together AI operates an AI-native cloud platform for open and custom AI models.
About Together AI
Together AI provides production AI infrastructure spanning inference, accelerated compute, model training and fine-tuning, and secure code sandboxes for AI development.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will lead the team that runs Kubernetes on bare-metal GPU infrastructure across multiple data centers. You will set technical direction, drive zero-touch provisioning of GPU nodes, own cluster lifecycle at scale, and make GPU scheduling and high-performance networking first-class. You will define SLOs, invest in observability, hire and grow the team, build a healthy on-call culture, and work with cross-functional partners to plan capacity and launch new hardware generations.
Requirements
- 8+ years in infrastructure, SRE or platform engineering, including 2+ years managing engineers
- Deep hands-on experience running Kubernetes in production at scale, ideally on bare metal
- Solid grounding in Linux systems, networking and hardware: kernels, drivers, BIOS/BMC, PXE boot and out-of-band management
- Experience building automation for fleet provisioning and lifecycle with tools such as Cluster API, Metal3, Ironic, MAAS, Ansible or Terraform
- Track record of owning production systems with on-call, SLOs and incident response
- Strong technical judgment
- Clear communication with engineers, leadership and external partners
Responsibilities
- Set the technical direction for running Kubernetes on bare metal across multiple data centers and regions
- Drive automated, zero-touch provisioning of GPU nodes from firmware and OS imaging through cluster join and validation
- Own cluster lifecycle at scale: upgrades, capacity expansion, node health checks, automated remediation and decommissioning
- Make GPU scheduling, topology-aware placement and high-performance networking (InfiniBand, RoCE, RDMA) first-class
- Define and meet SLOs for cluster availability, node bring-up time and GPU utilization
- Invest in observability across hardware, networking, Kubernetes and workloads
- Cut the time from racked hardware to customer-ready capacity
- Hire, manage and develop a team of infrastructure and platform engineers
- Run team planning, prioritization and delivery
- Build a healthy on-call culture with sustainable rotations and blameless postmortems
- Work with data center operations, networking, hardware vendors, inference, training and product teams to plan capacity and launch new hardware
- Represent platform needs to leadership and turn customer requirements into a roadmap
Benefits
- Startup equity
- Health insurance
- Other competitive benefits