Senior QA Engineer
Core
Building test and evaluation systems for AI voice agents that interact with patients and providers in healthcare and pharma, ensuring quality across speech recognition, LLMs, and telephony layers.
Role type
Senior QA Engineer (Voice Agents & LLMs)
Builds
Production-ready voice agents for health systems and pharma organizations
Domain
Healthcare / Pharma / AI Voice Agents / LLMs
Deliverable
production ML models
Required skills
Classical QA strategy, Voice agent testing, LLM system testing, Test automation (Python), Non-deterministic system testing, Release criteria definition, Red-teaming, System mindset
Preferred skills
IVR/Contact Center background, Audio/network condition testing, LLM evaluation tooling, HIPAA compliance, QA function setup, Performance testing, Spanish proficiency
Technologies
Python, CI/CD, Speech-to-text, Text-to-speech, Telephony systems, LLM frameworks
Responsibilities
Design and maintain test strategies for voice and LLM systems; Debug complex call failures across multiple layers (ASR, model, TTS, telephony); Define release criteria and regression suites for non-deterministic outputs; Conduct red-teaming and security testing for guardrails; Monitor production quality and identify model drift; Automate testing tooling and integrate into CI pipelines.
Seniority
Senior, hands-on IC