CareerPlan

Certification Assessment Specialist

San FranciscoFull-time2026-11-19 → 2026-11-20
Based in United States

Core

Design, build, test, launch, and maintain hands-on technical assessments and scoring workflows for OpenAI role-based certifications.

Role type

Senior IC assessment engineering specialist

Builds

Production assessment environments, scoring pipelines, and model-assisted scoring workflows

Domain

AI/ML certification and performance assessment

Deliverable

production ML models

Required skills

assessment design, scripting and automation, environment provisioning, scoring pipeline development, model-assisted scoring validation, end-to-end testing, integration testing, version control, accessibility implementation, defect investigation

Preferred skills

AI application development assessment, psychometric rubric calibration, live and remote certification launch

Technologies

scripting languages, assessment platforms, model-assisted scoring tools

Responsibilities

Translate assessment blueprints into realistic scenarios and hands-on tasks; define and build assessment environments with repeatable provisioning workflows; build pipelines to capture submissions and apply scoring rules; test and validate model-assisted scoring against human expert judgments; conduct end-to-end testing including dry runs and failure recovery; translate assessment needs into delivery integration requirements; prepare assessments for launch and document troubleshooting; maintain versioned tasks and configurations; investigate technical issues during delivery.

Seniority

Senior, hands-on IC