Senior Software Engineer AI Benchmarking
SecureBio is a U.S. 501(c)(3) nonprofit that develops pandemic-defense capabilities and evaluates AI-enabled biological risks.
Funding history
Investors
About SecureBio
SecureBio operates pathogen-agnostic early-warning work through deep metagenomic sequencing and builds evaluations and mitigations for biosecurity-relevant frontier-AI capabilities.
Skills
About the Role
You will build, scale, maintain, and improve biosecurity evaluations for frontier and open-source AI models. You will develop model adapters and internal assessment tools, troubleshoot agent behavior, contribute cloud infrastructure, conduct analyses, and support the public dashboard of AI biology capability trends.
Requirements
- 5+ years of software engineering experience in industry roles and/or open-source contributions
- Python fluency
- Docker proficiency
- AWS proficiency, particularly ECS
- Experience building tools that use or evaluate LLMs or AI agents
- Knowledge of coding practices and coding-agent use
- Code quality, readability, and human oversight of human- and AI-written code
Responsibilities
- Assess frontier and open-source AI models using biosecurity evaluations
- Build model adapters and troubleshoot unexpected agent behavior
- Build, scale, maintain, and improve biosecurity evaluations
- Improve evaluations using agent frameworks, jailbreaking methods, and elicitation strategies
- Develop internal tools for scalable capability assessments
- Contribute to cloud infrastructure for evaluations and analyses
- Contribute analyses to the public AI biology capabilities dashboard
Benefits
- Health, dental, and vision insurance
- Yearly mental health allowance
- Flexible work hours
- Unlimited paid time off
- Minimum 12 weeks of fully paid parental leave
- Fully paid conference sponsorships
- Yearly development allowance
- Employer 401(k) match up to 4% of total salary
Hiring Process
Written application review; possible short coding test and short screening interview; further interviews and/or paid trial tasks for high-performing candidates; up to two reference checks.