Senior QA Automation Engineer
Core
Lead QA Engineer owning quality for an AI voice virtual agent, focusing on testing conversational behavior, non-deterministic LLM outputs, and SIP/VoIP systems.
Role type
Senior Lead QA Engineer (AI Voice & Telephony)
Builds
AI voice virtual agent across LLM, speech, and telephony layers
Domain
AI/ML, Telecommunications, Voice AI
Deliverable
production ML models | product features
Required skills
Test strategy design, SIP/VoIP troubleshooting, Python automation, non-deterministic system testing, quality metrics definition, TMS tool usage, CI/CD integration
Preferred skills
LLM evaluation frameworks, ASR/TTS quality assessment, adversarial testing techniques
Technologies
Python, pytest, Playwright, JsSIP, WebRTC, Allure TestOps, Jira, Jenkins, GitHub Actions, Grafana
Responsibilities
Own end-to-end test strategy for AI voice agent; Design test scenarios for complex conversational behavior; Build evaluation framework for non-deterministic output; Adversarially test agent against hallucinations and security risks; Prepare and provision SIP devices for automation; Diagnose failures across LLM, ASR, TTS, and media layers; Extend automated coverage and integrate into CI/CD; Define and track core quality metrics; Maintain documented test coverage with traceability; Communicate quality risks to stakeholders.
Seniority
Senior, hands-on IC with leadership responsibilities
