Principal Researcher, Benchmarks, Evaluation, and Standards
Independent nonprofit, multi-stakeholder partnership that convenes academia, civil society, industry, media, and philanthropy to create research, recommendations, tools, and practical guidance for responsible AI.
Funding history
Investors
About Partnership on AI
Partnership on AI (PAI) is an independent 501(c)(3) nonprofit partnership focused on ensuring AI advances positive outcomes for people and society. It convenes diverse stakeholders, conducts research, develops actionable responsible-AI guidance, informs public policy, and advances public understanding.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will lead technical strategy for benchmarking, evaluation, and standards work across AI governance and deployment. You will develop and operationalize guidance, conduct and publish research, build partnerships, represent the organization externally, and translate rigorous findings into practical recommendations for varied audiences.
Requirements
- Eight or more years of relevant technical experience in AI, human-computer interaction, trust and safety, or a relevant interdisciplinary field
- Fluency in technical and sociotechnical domains
- Strong, accessible writing for different audiences
- Experience translating guidance into evaluations, benchmarks, standards, audit criteria, certifications, or professional training
- Ability to convene stakeholders with conflicting interests and varied disciplinary backgrounds
- Experience representing an organization at events with executives and leaders
- Ability to set objectives and drive multistakeholder strategy autonomously
Responsibilities
- Develop PAI's perspective and strategy for the evaluation and benchmarking ecosystem
- Assess infrastructure needed to develop and run evaluations
- Advance shared industry guidance through benchmarks, evaluations, and technical standards
- Provide technical leadership for PAI focus areas
- Support technical efforts from Policy and International teams
- Design sociotechnical solutions and implementation pathways
- Ensure guidance is technically feasible and implementable as evaluations or standards
- Support testing and sandboxing exercises related to guidance
- Design an approach to sustain benchmarking, evaluation, and standards work, including partnership quality control
- Author or co-author peer-reviewed papers, policy briefs, and accessible reports
- Initiate and lead collaborative research partnerships
- Represent PAI at conferences and other venues
- Manage relationships with benchmarking, standards, evaluation, and assurance organizations
- Collaborate on earned-revenue models such as licensing and credentialing contributions
- Translate findings into clear narratives for diverse audiences
- Support communications that increase adoption of guidance
- Represent work to standards bodies, policymakers, media, funders, and the public
Benefits
- Twenty vacation days, three personal reflection days, and sick and family leave above industry standards in the US
- High-quality PPO and HMO health insurance plans, many fully covered in the US
- Dental and vision insurance fully covered in the US
- Up to a 7% immediately vested 401(k) match in the US
- Automatic cell phone reimbursement of $75 per month
- Annual professional-development funds
- $150 per month for co-working space access
- Regular team lunches and focused work days
- Opportunities to attend AI-related conferences and collaborate with partners