CareerPlan

Agentic AI Systems Evaluation Engineer

Houston, TXFull-time$131k–$205k2026-10-20 → 2026-11-19

Core

Evaluate AI software to ensure it delivers designed customer experiences and system performance by connecting user stories to measurable outcomes.

Role type

Senior IC systems evaluation engineer (AI inference)

Builds

Benchmarking frameworks, evaluation tools, and actionable feedback for AI workloads

Domain

PC systems and AI inference

Deliverable

production ML models

Required skills

AI inference workloads, benchmarking methodologies, system-level performance, Python/C++ programming, model formats (ONNX/GGUF), telemetry analysis

Preferred skills

Windows/Linux PC evaluation, attention mechanisms/KV caching/quantization, heterogeneous hardware (CPU/GPU/NPU), statistical analysis

Technologies

Python, C++, ONNX, GGUF

Responsibilities

Translate user stories into measurable success criteria and evaluation scenarios; Design repeatable benchmarking frameworks for AI workloads; Compare model formats, runners, and hardware configurations; Analyze inference behavior (latency, token rate, memory); Evaluate system responsiveness under concurrent workloads; Interpret benchmark data to identify meaningful improvements or regressions; Build automation tools for reproducible evaluations

Seniority

Senior, hands-on IC