Technical Legal Engineer - Evals
Core
Evaluate, stress-test, and improve AI systems used in legal work by reviewing AI-generated outputs for accuracy, completeness, and legal reasoning.
Role type
Senior IC legal engineer (evals)
Builds
High-quality evaluation datasets, benchmark cases, gold-standard legal answers, and scoring rubrics for AI systems
Domain
Legal technology + Large Language Models (LLMs)
Deliverable
production ML models
Required skills
Legal training and practice experience, issue-spotting, quality control, analytical rigor, ability to translate legal feedback into actionable guidance for non-lawyers
Preferred skills
Knowledge of tech products, AI, and LLM evaluations
Technologies
LLMs, AI systems
Responsibilities
Review AI-generated legal outputs for accuracy and jurisdictional correctness; Design and build evaluation datasets and benchmark cases; Identify recurring legal errors and failure patterns; Collaborate with legal engineers and product managers to ensure AI outputs meet professional standards
