Senior AI QA Engineer
Core
Build validation frameworks to ensure clinical insights are accurate, safe, and reliable using automated grading and data engineering.
Role type
Senior AI QA Engineer (ML Validation)
Builds
Automated model grading systems, test harnesses, and data pipeline integrity checks for clinical AI outputs.
Domain
Healthcare / AI/ML Validation
Deliverable
production ML models
Required skills
Python, AI/ML validation, automated model grading, agentic AI tools, test automation frameworks, API testing, database validation
Preferred skills
Clinical domain knowledge (FHIR, HL7, SNOMED CT), healthcare compliance (HIPAA), statistical metrics (sensitivity, specificity, F1-scores)
Technologies
LangSmith, Arize Phoenix, MLflow, Cursor, Windsurf, Claude Code, PyTest, Cypress, Playwright, Selenium, Postman, RestAssured
Responsibilities
Architect multi-layered validation frameworks combining deterministic checks and semantic metrics; Develop automated systems for model grading and hallucination detection; Leverage agentic AI tools to prototype test harnesses and red-team clinical logic; Execute integration and regression tests for data-heavy backend processes; Collaborate with Data Scientists to define Ground Truth datasets and evaluation rubrics; Perform root cause analysis on system failures.
Seniority
Senior, hands-on IC