Staff+ Software Engineer, Safeguards Evals

Skills

About the Role

You'll design experiments to improve how model behavior and agentic safety systems are evaluated, build datasets representing real abuse, own an eval harness for an agentic investigation system, and ship methods into training, regression and release pipelines. You'll also build tooling so policy experts can author and run evals.

Requirements

• Experience building and maintaining data pipelines
• Experience working with LLMs and agentic systems
• Strong data analysis skills
• Ability to move between research prototyping and production-quality code
• Ability to translate ambiguous problems into testable experiments
• Care about AI safety

Responsibilities

• Design and run experiments to improve evaluation quality
• Build and own the evaluation harness for an agentic investigation system
• Construct RL environments to improve Claude's safety investigation capabilities
• Construct high-quality eval datasets representing real-world misuse
• Collaborate with Policy and Enforcement to turn harm patterns into evaluations
• Ship research into evaluation, regression and release pipelines
• Build tooling for policy experts to author and run evaluations
• Surface findings to research and training teams

Benefits

• Competitive compensation and benefits
• Optional equity donation matching
• Generous vacation and parental leave
• Flexible working hours
• Visa sponsorship