AI Engineering Lead, Evals and Systems Integration
Core
Own and evolve the evaluation framework and systems integration layer for a full-stack agentic decisioning platform serving large organizations.
Role type
Senior IC AI Engineering Lead (Evals & Systems Integration)
Builds
Integrated suite of agentic decisioning tools for large organizations
Domain
Applied AI / Agentic Systems / Enterprise Consulting
Deliverable
production ML models
Required skills
Evaluation framework design, systems integration testing, statistical reasoning, experimental design, LLM evaluation, agentic workflow reliability, entity resolution, data ontology validation, deployment-readiness standards
Preferred skills
Experience with tool and context protocols (MCP), building evaluation harnesses, working with messy enterprise data
Technologies
MCP, LLMs, Knowledge Graphs, Agentic Workflows
Responsibilities
Define quality standards, scoring rubrics, and deployment-readiness thresholds for the full system; Design and maintain evaluation datasets and regression test suites; Engage with consulting teams to translate client feedback into structured evaluation improvements; Present quality and reliability findings to senior client stakeholders.
Seniority
Senior, hands-on IC
