[Agentic AI] Systems Evaluation Engineer
Core
Define measurement methodology and benchmarks to quantify customer outcomes, platform performance, and AI workload effectiveness.
Role type
Senior Systems Evaluation Engineer
Builds
Technical solutions to help developers deliver faster with modern software and application development tools
Domain
Software Engineering / AI Systems Evaluation
Deliverable
dashboards & analysis
Required skills
Benchmarking methodologies, Analytical skills, Software architecture, Programming languages, Strategic planning
Preferred skills
AI systems evaluation, Telemetry analytics, Linux/Windows validation, Vendor management
Technologies
AWS, Azure, Kubernetes, Docker, C++, Java, Python, JavaScript, React.js, SQL, Microservices, APIs, DevOps, Automation
Responsibilities
Define success metrics, Create benchmarking frameworks, Analyze customer outcomes, Produce executive performance reports, Contribute to technology strategy and engineering roadmaps, Supervise external engineering vendors
