Senior AI QA Engineer
Core
Architect and implement multi-layered validation frameworks to ensure clinical insights are accurate, safe, and reliable using automated grading methodologies.
Role type
Senior AI QA Engineer (Clinical AI Validation)
Builds
Automated model grading systems, test harnesses, and data pipeline integrity checks for medical outputs.
Domain
Healthcare / AI/ML Validation
Deliverable
production ML models
Required skills
System architecture, Python (expert), AI/ML validation metrics, Agentic AI tools, Test automation frameworks, API & database testing
Preferred skills
Clinical domain knowledge (FHIR, HL7), Healthcare compliance (HIPAA), Statistical analysis
Technologies
BERTScore, ROUGE, LangSmith, Arize Phoenix, MLflow, Cursor, Windsurf, Claude Code, PyTest, Cypress, Playwright, Selenium, Postman, RestAssured
Responsibilities
Design structured testing strategies combining deterministic checks and semantic metrics; Develop automated systems for faithfulness, safety, and hallucination detection; Prototype complex test harnesses using agentic AI tools; Execute integration and regression tests for data-heavy backend processes; Collaborate with Data Scientists to define Ground Truth datasets; Perform root cause analysis on system failures
Seniority
Senior, hands-on IC