STEM & Technical Specialist – Rubric Design
Core
Create expert-level technical questions, design detailed evaluation rubrics, and evaluate AI-generated solutions to improve frontier AI models.
Role type
Senior IC STEM & Technical Subject Matter Expert (AI Evaluation)
Builds
Training data (questions, rubrics, reference answers) for foundation-model labs and AI companies
Domain
AI/ML + STEM (Mathematics, Physics, Chemistry, Engineering, Computer Science)
Deliverable
production ML models
Required skills
STEM domain expertise, Technical Reasoning, AI Evaluation, Rubric Design, Rubric Writing, Benchmarking, Model Evaluation, Question Writing, Expert Review
Preferred skills
Exam writing, Academic grading, Peer review, Research review, Technical QA, Standards development, Assessment design, AI evaluation, RLHF, SFT, Benchmarking, Data annotation, Prompt design, LLM evaluation
Technologies
LLMs, RLHF, SFT
Responsibilities
Create expert-level technical questions, Design evaluation rubrics, Define partial-credit standards, Evaluate AI-generated solutions, Identify hidden reasoning errors, Critique peer rubrics, Develop reference content, Support AI training, Maintain technical rigor