Machine Learning Research Scientist, Evaluations
Core
Develop rigorous evaluations and diagnostic methods to identify, characterize, and diagnose failure modes in frontier LLMs and Agents across text and multimodal modalities.
Role type
Research Scientist specializing in LLM evaluation and failure analysis
Builds
Benchmarks, evaluation methods, and diagnostic frameworks for generative AI models
Domain
Generative AI, Large Language Models (LLMs), Multimodal AI
Deliverable
production ML models
Required skills
LLM evaluation, benchmark development, failure mode analysis, root cause analysis (RCA), reinforcement learning, large-scale model fine-tuning, post-training techniques (SFT, RLHF, reward modeling), deep learning
Preferred skills
Published research in top-tier AI conferences, experience in customer-facing roles
Technologies
LLMs, Agents, RLHF, SFT, Reward Modeling
Responsibilities
Analyze model behavior to identify capability gaps, reasoning errors, robustness, and alignment issues; Design and build benchmarks measuring LLM capabilities; Apply post-training expertise to connect failures to data and training interventions; Publish research findings in top-tier AI conferences
Seniority
Senior, hands-on IC