Technical Operations Analyst, Automation & Evaluations
OpenAIVisit OpenAI website
AI research and deployment company building and deploying frontier AI products, including ChatGPT, Codex, and its API platform.
109 current maintainers73 active leads37 new active leads109 lead step-downs11 early lead departures
San Francisco, United States
Projects
About OpenAI
OpenAI’s mission is to ensure artificial general intelligence benefits all of humanity. It consists of the nonprofit OpenAI Foundation and OpenAI Group, a public benefit corporation governed by the Foundation.
Skills
Candidate Availability
About the Role
You will partner with teams across User Safety & Risk Operations to support their data, analysis, measurement and automation needs. You will build tools, datasets, dashboards and workflows, evaluate them against expert-reviewed examples, and hand them off with clear ownership. The role may involve exposure to sensitive content.
Requirements
- Strong SQL skills
- Python for data analysis and tool building
- Experience with messy operational data
- Ability to turn loosely defined requests into clear problems
- Experience building datasets, dashboards, scripts or API integrations
- Ability to define metrics
- Understanding of workflow automation trade-offs
- Experience designing evaluations or quality checks
- Nice to have: experience transferring prototypes into scaled operations
Responsibilities
- Define the questions teams need answered and identify work that could benefit from automation
- Use SQL and Python to investigate operational problems and make recommendations
- Build and maintain datasets and dashboards
- Define metrics to assess quality and impact of work
- Build scripts and API integrations to automate recurring tasks using LLMs where useful
- Evaluate automated workflows against expert-reviewed examples
- Hand off workflows with clear ownership, maintenance instructions and feedback loops
