Research, Post-Training Evals
Core
Develop reliable model evaluations for research signals spanning evaluation creation, usability, auditing, and efficiency.
Role type
Researcher (AI Evaluation & Post-Training)
Builds
Internal evaluations, research signals, usability metrics, benchmark auditing methodologies, and agentic evaluation environments.
Domain
Artificial Intelligence / Machine Learning Research
Deliverable
production ML models
Required skills
Evaluation design, benchmark creation, grader reliability analysis, human evaluation coordination, Python programming, deep learning frameworks, research methodology, technical writing
Preferred skills
LLMs, post-training, reinforcement learning, agentic systems, evaluation auditing, LLM-judges, harness development, preference evaluation, bias detection
Technologies
Python, PyTorch, TensorFlow, JAX
Responsibilities
Create internal evaluations and research signals for model capabilities; Develop usability evaluations for real-world workflows; Improve evaluation robustness and grader reliability; Build benchmark auditing methodologies; Develop specialized agentic evaluation environments and user simulators; Develop evaluations for personalized preferences and biases.
Seniority
Mid-Senior level Researcher