Research Engineer, Benchmarks & Evaluation
Core
Build benchmarks, graders, and RL environments from real finance workflows to evaluate and improve LLMs for financial tasks.
Role type
Senior IC research engineer (LLM benchmarks & evaluation)
Builds
Versioned benchmark suites, programmatic graders, and RL environments for long-horizon agents
Domain
AI research + Finance
Deliverable
production ML models
Required skills
LLM benchmarking, Python software engineering, statistical analysis, post-training (SFT/RL), SQL, large dataset handling
Preferred skills
RL environment design, eval framework contribution, published benchmark research, domain expertise in accounting/finance
Technologies
Python, SQL, LLM-as-judge, RL environments
Responsibilities
Build benchmarks from real workflows (agent traces, expert labels), own the benchmarking effort and reporting, design graders (rubrics, programmatic checks), close the loop with post-training (reward functions, RL environments), publish external benchmarks
