AI Engineer - Data and Evals
Core
Building reliable data and evaluation systems for Glean's AI products, including evalsets, benchmarks, and simulation environments to measure and improve agent performance.
Role type
Senior IC AI Engineer (Data & Evaluation)
Builds
Evalsets, benchmarks, agentic LLM verifiers, experiment pipelines, trace capture/replay dashboards, and enterprise simulation environments.
Domain
Enterprise AI / Agentic AI / Evaluation Science
Deliverable
production ML models
Required skills
Go, Python, backend engineering, API design, data modeling, distributed pipelines, SQL, LLM eval methodologies, LLM-as-a-judge, synthetic data generation, CI/CD, observability (OpenTelemetry), statistical analysis.
Preferred skills
Simulation environments, model post-training, TypeScript.
Technologies
Go, Python, TypeScript, OpenTelemetry, LLMs, SQL.
Responsibilities
Build evalsets and benchmarks from expert-curated, production, synthetic, and failure data; build and scale the eval platform and agent observability; build agentic LLM verifiers for correctness and task success; run evals at every stage from CI to production traffic; build enterprise simulation environments; diagnose failures and turn results into concrete fixes.