Research Engineer / Scientist, Data & Evals
Core
Build data and measurement systems for post-training, benchmarking, and agent capability efforts in legal AI, including synthetic data pipelines and evaluation design.
Role type
Senior IC research engineer (data & evals)
Builds
Synthetic data pipelines, evaluation systems, and capability benchmarks for legal reasoning and agentic workflows
Domain
Legal technology + Machine Learning
Deliverable
production ML models
Required skills
Python, data pipeline engineering, ML evaluation system design, statistical reasoning, experimental design, rubric creation, data quality management
Preferred skills
synthetic data generation, LLM evaluations, automated raters, preference data, legal domain knowledge
Technologies
Python
Responsibilities
Build synthetic data pipelines for training and evaluation; Design evaluations and verifiers reflecting real legal work; Maintain capability benchmarks like Legal Agent Benchmark; Define quality metrics and grading mechanisms with legal experts; Ensure data quality, provenance, and contamination safeguards; Partner with researchers to investigate model failures and guide training decisions
