We launched recruiters.rip, let your AI apply for you. First month free with code onemonthfree

Certification Assessment Specialist

AI research and deployment company building and deploying frontier AI products, including ChatGPT, Codex, and its API platform.

105 current maintainers70 active leads37 new active leads111 lead step-downs12 early lead departures
San Francisco, United States
About OpenAI

OpenAI’s mission is to ensure artificial general intelligence benefits all of humanity. It consists of the nonprofit OpenAI Foundation and OpenAI Group, a public benefit corporation governed by the Foundation.

View jobs by OpenAI

Skills

Candidate Availability

About the Role

You will design, build, test, launch and maintain hands-on assessments behind an OpenAI technical role-based certification. You will turn blueprints and rubrics into realistic tasks, build assessment environments and evidence-capture and scoring workflows, validate model-assisted scoring with human reviewers, and work with engineering and delivery partners to launch and maintain assessments.

Requirements

  • Experience designing and building performance-based hands-on assessments, labs or simulations for technical roles
  • Experience using scripting and automation to build and troubleshoot assessment environments
  • Experience at a technology company in a fast-paced environment
  • Ability to use a blueprint and rubric to design realistic tasks
  • Experience building workflows that capture submissions and produce traceable scores
  • Experience implementing or validating model-assisted scoring against expert judgments
  • Experience working with external technical experts or delivery providers
  • Understanding of accessibility and protecting confidential assessment content
  • Clear communication of technical tradeoffs

Responsibilities

  • Translate assessment blueprints into realistic scenarios, candidate instructions and hands-on tasks
  • Define and build assessment environments with repeatable provisioning and reset workflows
  • Build and maintain pipelines that capture submissions, apply scoring rules and route results for review
  • Build and test model-assisted scoring with human reviewers
  • Test assessments end to end including dry runs and pilots
  • Define requirements for delivery integrations and partner with engineering and providers
  • Prepare assessments for launch and document setup and troubleshooting
  • Maintain versioned tasks, environments, scoring configurations and test fixtures
  • Investigate and fix technical issues raised during delivery or review

Benefits

  • Equity
  • Performance-related bonus
  • Medical, dental and vision insurance
  • Health Savings Account employer contributions
  • Health FSA, Dependent Care FSA and commuter pre-tax accounts
  • 401(k) with employer match
  • Paid parental leave
  • Paid medical and caregiver leave
  • Flexible PTO
  • 13+ paid company holidays and company office closures
  • Paid sick or safe time
  • Mental health and wellness support
  • Employer-paid basic life and disability coverage
  • Annual learning and development stipend
  • Daily meals in offices and meal delivery credits
  • Relocation support
  • Charitable donation matching and wellness stipends