Software Engineer, Agent Optimization & Evaluation
Core
Design and build an optimization harness for evaluating, benchmarking, and improving AI agents across software engineering workflows.
Role type
Senior IC machine-learning engineer (AI agents & evaluation systems)
Builds
Optimization harnesses, evaluation environments, model routing/orchestration strategies, and experimentation frameworks for AI agents.
Domain
Software Engineering + AI/LLM Agents
Deliverable
production ML models
Required skills
Large-scale production software systems design, AI agent evaluation/benchmarking, modern LLMs, system optimization (quality/latency/reliability/cost), software engineering fundamentals, technical direction in ambiguous environments.
Preferred skills
Experience with Reddit's flexible, model-agnostic agent platform strategy, cross-functional collaboration across engineering/infrastructure/platform organizations.
Technologies
LLMs, AI agents, model routing, orchestration, synthetic task generation, human-in-the-loop feedback systems.
Responsibilities
Design model routing and orchestration strategies; define metrics and experimentation frameworks; build tooling to analyze agent behavior and investigate failures; partner with Platform Engineering and Infrastructure teams.
Seniority
Senior, hands-on IC
