QA & Support Engineer — AI Voice Platform

## About the role We're looking for a QA & Support Engineer to own product quality, AI behaviour testing, production monitoring, and customer support operations across our AI voice platform. This is not a manual click-through QA role. You'll test real AI agents, validate live call flows, write automated test cases, monitor production issues, reproduce bugs, communicate with customers, and work closely with engineering to improve system reliability. You'll work across Python backend systems, LLM workflows, voice AI pipelines, campaign flows, post-call analysis, dashboards, logs, monitoring tools, and customer-reported issues. ## What You'll Work On ### Feature Testing & QA - Write and maintain automated test suites for APIs, agent workflows, and campaign logic - Test backend APIs, webhooks, queues, background jobs, and async workflows - Validate end-to-end campaign flows including call lifecycle, contact progression, retries, and post-call analysis - Test dashboard features including reports, filters, analytics, call data, and real-time session state - Perform regression testing for every release, including AI model, prompt, and workflow changes - Run smoke tests, integration tests, and load/stress tests for high-concurrency call scenarios ### AI Behaviour Testing - Test whether AI agents follow instructions, stay on script, handle objections, and complete goals - Validate tool calling, workflow execution, memory injection, fallback handling, and edge cases - Test LLM outputs for accuracy, consistency, hallucinations, and instruction-following - Build golden test cases for agent conversations with expected inputs, expected outputs, and pass/fail criteria - Catch regressions after prompt changes, model changes, workflow updates, or backend changes - Validate AI evaluation scores, post-call summaries, sentiment analysis, and outcome detection ### Voice AI Pipeline Testing - Test real-time voice flows across speech-to-text, LLM, and text-to-speech stages - Validate call setup, audio routing, turn-taking, interruption handling, and response timing - Test voice AI tools and APIs such as OpenAI, Gemini, Deepgram, ElevenLabs, or similar platforms - Monitor latency across speech recognition, LLM response, and voice synthesis - Identify failed calls, dropped calls, timeout issues, silence handling problems, and audio quality issues ### Monitoring & Observability - Monitor production dashboards for uptime, error rates, latency, queue health, and campaign KPIs - Set up and maintain alerts for failed calls, API errors, timeout spikes, queue failures, and agent crashes - Read and debug logs, stack traces, structured errors, and API responses - Identify production failure patterns before they become customer complaints - Track AI quality drops, latency spikes, and unexpected agent behaviour - Work with engineering to improve logging, metrics, tracing, and monitoring coverage ### Production Support - Own customer support communication for technical issues - Triage customer-reported bugs, reproduce issues, and identify root causes - Distinguish between user errors, configuration issues, data issues, and product bugs - Escalate issues to engineering with clear logs, steps to reproduce, screenshots, recordings, and expected vs actual behaviour - Write clear customer-facing updates during bugs, incidents, and issue resolution - Build and maintain internal runbooks, QA checklists, support docs, and known-issue documentation ### Backend & Data Validation - Validate data correctness across databases, dashboards, call records, campaign analytics, and post-call outputs - Write SQL or NoSQL queries to verify system behaviour and debug data issues - Test webhook delivery, retry logic, async processing, and eventual consistency - Validate contact status updates, call outcomes, lead progression, and analytics calculations - Work with Python-based backend systems, APIs, background jobs, and cloud-native deployments ## You're a Fit If You - Have experience writing automated tests for APIs and backend systems - Can test more than UI flows — APIs, async jobs, webhooks, queues, and data pipelines - Can read logs, trace errors, debug failures, and reproduce issues independently - Understand async systems, retries, webhooks, eventual consistency, and background workers - Are comfortable testing AI-powered features, LLM outputs, and agent behaviour - Have worked with Python, JavaScript/TypeScript, or similar languages for test automation - Have experience with monitoring, alerting, production debugging, or technical support - Can communicate clearly with both customers and engineering teams - Are detail-oriented, structured, and comfortable working in a fast-moving product environment ## Required Skills - API testing - Automated test writing - Python or JavaScript/TypeScript for test automation - Backend testing - Integration testing - Regression testing - Smoke testing - Load testing basics - Debugging logs and stack traces - SQL or NoSQL data validation - CI/CD testing workflows - Webhooks, queues, retries, and async systems - Production monitoring and alerting - Customer support and issue triage - Clear writing