Senior Quality Engineer - AI
Core
Define and execute quality strategies for AI-powered product experiences (LLMs, RAG, agents), building evaluation systems and translating feedback into product improvements.
Role type
Senior IC Quality Engineer (AI/ML)
Builds
Automated and human evaluation workflows, quality pipelines, and measurable AI output standards for student housing software.
Domain
SaaS B2B/B2B2C, Student Housing, AI/ML
Deliverable
production ML models | product features
Required skills
AI quality strategy, LLM evaluation, RAG validation, prompt engineering, agent workflow testing, human reviewer calibration, shift-left quality practices, data-driven quality management, test automation, observability
Preferred skills
LLM judges, agent observability platforms, safety/bias/compliance evaluation, production monitoring (drift detection), domain-specific AI products
Technologies
LLMs, RAG, prompts, agents, automated eval tooling
Responsibilities
Lead end-to-end AI quality strategy for LLMs and agent workflows; Coach human reviewers and SMEs on evaluation criteria; Define AI quality rubrics and standards; Partner with engineering to identify AI risks (hallucination, grounding); Build repeatable evaluation approaches for model/prompt changes; Translate failure patterns into product improvements.
Seniority
Senior, hands-on IC with leadership influence
