Staff Software Engineer, Model Infrastructure

Skills

About the Role

You will lead the design and development of the systems that power every AI request. You will build the Unified Model Controller and Model Selector, develop provisioning, capacity management and failover across AI providers, integrate new model providers, improve observability and cost reporting, and mentor engineers.

Requirements

• 10+ years of software engineering experience building large-scale distributed systems
• Experience designing and operating highly available production services
• Strong programming skills in Go, Java, Python, Rust or C++
• Deep understanding of distributed systems, cloud infrastructure, networking and observability
• Experience leading technical projects across multiple teams
• Strong communication and collaboration skills
• Nice to have: AI infrastructure, LLM serving, model routing, Kubernetes, SRE, Kafka, Spark, Flink, Airflow, Iceberg, GPU infrastructure

Responsibilities

• Lead design and implementation of the Model Infrastructure platform
• Build systems for high availability and low latency AI inference
• Design and improve the Unified Model Controller and Model Selector platform
• Develop model provisioning, capacity management, failover and traffic engineering across AI providers
• Integrate new model providers and maintain provider APIs and SDKs
• Improve observability through dashboards, alerting, token usage analytics and cost reporting
• Partner with Product Engineering on model launches, experimentation and monitoring
• Drive infrastructure efficiency through capacity planning and utilization optimization
• Collaborate with AI Research on infrastructure for model evaluation, training and deployment
• Lead cross-functional technical initiatives and mentor engineers

Benefits

• Equity