Backend Software Engineer (Evals)
Core
Design and build evals infrastructure to measure the quality of OpenAI's support automation systems and knowledge creation.
Role type
Senior Backend Software Engineer (ML/LLM Evals)
Builds
Robust backend services, APIs, and continuous evaluation monitoring frameworks for intelligent automation.
Domain
AI Research & Deployment / LLM Evaluation Infrastructure
Deliverable
production ML models
Required skills
Backend engineering, distributed systems design, API development, data pipeline scaling, LLM evaluation methods, multi-agent workflow design, production eval creation
Preferred skills
Experience with Python, FastAPI, Postgres, building AI agents, long context handling, tool use patterns
Technologies
Python, FastAPI, Postgres
Responsibilities
Design reliable and reproducible eval pipelines; build infrastructure for regression and drift monitoring; maintain backend services for intelligent automation; integrate and structure data for AI workflows; collaborate with data science and research teams; own the full development lifecycle of backend systems.
Seniority
Senior, hands-on IC
