Software Engineer (AI Quality & Reliability)
Core
Build automated evaluation, regression, and test infrastructure for voice AI agents and healthcare workflows to ensure reliable releases.
Role type
Senior IC software engineer (AI quality & reliability)
Builds
Automated evaluation frameworks, test harnesses, versioned datasets, and quality dashboards for voice agents and integrations
Domain
Healthcare + AI/ML quality engineering
Deliverable
production ML models
Required skills
Python, TypeScript, test automation, CI/CD pipelines, LLM evaluation, API testing, distributed systems testing, failure reproduction, regression testing
Preferred skills
Streaming audio, telephony, EHR integrations, privacy-safe data handling, healthcare workflows
Technologies
Python, TypeScript, CI/CD tools, voice AI platforms
Responsibilities
Build shared evaluation frameworks and test harnesses; create automated end-to-end regression coverage; evaluate AI behavior including latency and task completion; produce release-readiness evidence and maintain quality dashboards; reproduce defects and add durable regression cases; integrate automated checks into CI/CD and reduce flaky tests
Seniority
Senior, hands-on IC