Member of Technical Staff (Software Engineer, Enterprise Agent Evals & Efficiency)
Core
Build evaluation environments, infrastructure, and strategies to measure and improve the reliability, efficiency, and cost-effectiveness of enterprise AI agents.
Role type
Senior IC machine-learning engineer (agent evaluation & efficiency)
Builds
Enterprise evaluation environments, automated graders, model routing strategies, and efficiency optimizations for AI agents.
Domain
AI Agents, Enterprise Automation, LLM Infrastructure
Deliverable
production ML models
Required skills
System design, Python/Go/Rust, LLM agent development, evaluation infrastructure design, statistical analysis, cross-team leadership
Preferred skills
Experience with multi-step workflows, tool use, context management, caching strategies
Technologies
Python, Go, Rust, LLMs, AI Agents
Responsibilities
Build realistic enterprise evaluation environments with complex workflows; Own evaluation infrastructure and automated graders; Analyze agent trajectories to identify root causes and ship improvements; Develop reusable skills and improve tool descriptions; Build model-selection and routing strategies; Improve agent efficiency via context management and caching; Build regression gates and launch-readiness signals.
Seniority
Senior, hands-on IC
