Member of Technical Staff Software Engineer Enterprise Agent Evals and Efficiency

Skills

About the Role

You will build realistic enterprise evaluation environments, task suites, and automated graders for agents. You will diagnose failures and improve agent harnesses, tools, skills, model routing, context management, caching, and regression gates to increase task success while reducing latency and cost.

Requirements

• 5+ years of software engineering or machine learning engineering experience
• System design
• Scalable infrastructure
• Python, Go, or Rust
• LLM application or agent development
• Tool use
• Multi-step workflows
• Evaluation infrastructure, benchmark design, or automated grading
• Agent harnesses, tool interfaces, or reusable skills
• Test environments or simulations
• Model routing or inference efficiency
• Context management
• Caching
• Experimental and statistical judgment

Responsibilities

• Build realistic enterprise evaluation environments with connected applications, datasets, roles, permissions, and reproducible state
• Own evaluation infrastructure, task suites, and automated graders
• Analyze agent trajectories and production failures and ship reliability improvements
• Develop reusable skills and improve tool descriptions, schemas, and result handling
• Build and evaluate model-selection and routing strategies
• Improve context management, skill loading, caching, and execution efficiency
• Build regression gates and launch-readiness signals
• Validate that evaluation gains improve production outcomes

Benefits

• Equity
• Health insurance
• Dental insurance
• Vision insurance
• Retirement benefits
• Fitness benefits
• Commuter accounts
• Dependent care accounts
• Regionally tailored benefits for eligible international employees