Software Engineer Benchmarking
Epoch AI is an independent, data-first AI research nonprofit that publishes research, data, and benchmarks on AI progress, infrastructure, capabilities, and societal impacts.
Funding history
About Epoch AI
Epoch AI investigates the trajectory of artificial intelligence by tracking drivers and bottlenecks, measuring model capabilities, and forecasting AI’s impacts. It operates public datasets and benchmarking work, and also provides commissioned research, model evaluations, and consultations.
Skills
About the Role
You will run and maintain benchmarking infrastructure, integrate AI providers, and implement existing and new benchmarks. You will contribute robust code, prototype evaluation ideas, support internal experiments, and collaborate with researchers and analysts to deliver accurate evaluation outputs.
Requirements
- More than two years of professional software engineering experience building and maintaining complex systems
- Ability to contribute robust, maintainable code
- Ability to work deeply in existing codebases and infrastructure
- Ability to generate ideas for benchmarks, experiments, and projects
- Professional-level English proficiency
Responsibilities
- Implement AI benchmarks in the evaluation infrastructure
- Develop the benchmark suite for new model releases
- Contribute to new benchmark development
- Pitch and prototype benchmark and experiment ideas
- Collaborate with researchers, analysts, and engineers
- Ensure evaluation data and outputs are accurate and integrated into research products
Benefits
- Fully remote environment with flexible work hours and schedules for most roles
- Global health insurance, life insurance, and pension plan where applicable
- Protected 30 days of annual PTO with no specific PTO limit
- Unlimited personal and sick leave
- Up to six months of combined paid and unpaid parental leave for permanent staff
- Flexible equipment, productivity-tool, and learning expense policy
- Access to Berkeley offices with paid meals, snacks, and gym access