AI Evaluation Engineer - Proofline
Core
Build evaluation harnesses, test infrastructure, and release gates to verify reliability and quality of AI-facing features in production.
Role type
Senior IC AI Evaluation Engineer (Systems Verification)
Builds
Evaluation scaffolding, API test infrastructure, and release quality gates for AI assistants
Domain
AI/LLM applications and production engineering
Deliverable
production ML models
Required skills
Python, TypeScript, API-level testing, Playwright, Cypress, LLM evaluation, non-deterministic system evaluation
Preferred skills
SDET/QA automation background, ML evaluation experience
Technologies
Python, TypeScript, Playwright, Cypress
Responsibilities
Build and maintain evaluation harnesses for AI-facing features; Own end-to-end and API test infrastructure; Design and execute host-behavior probes across diverse AI assistants; Gate production releases through UAT, regression analysis, and quality reporting
Seniority
Senior, hands-on IC