Sr. Software Development Engineer in Test (Agentic)
Core
Own end-to-end quality for agentic AI features, workflows, and orchestration flows, including strategy, development, execution, and release qualification.
Role type
Senior Software Development Engineer in Test (Agentic QA)
Builds
Automated test infrastructure, evaluation frameworks, and CI/CD integrations for AI Voice Agent services
Domain
AI/LLM-driven SaaS, agentic AI, voice and digital customer experience
Deliverable
production ML models | product features
Required skills
Python, Java, JavaScript, distributed cloud-native SaaS testing, API testing, LLM testing, probabilistic testing, CI/CD pipeline integration, LLM observability, performance/load testing, AI evaluation techniques
Preferred skills
Prompt engineering, model selection, human-in-the-loop validation, non-deterministic system testing
Technologies
Python, Java, JavaScript, Jenkins, GitHub Actions, Google Cloud Platform, Cloud Run, App Engine, Kubernetes, Datastore, Redis, ElasticSearch, Vue3, React, LiveKit
Responsibilities
Design and build automation tooling for AI/LLM-driven systems; Develop and maintain evaluation frameworks to measure response quality and hallucination rates; Integrate AI quality checks into CI/CD pipelines; Build tooling for LLM observability and debugging; Design and execute performance and load tests for AI services; Mentor engineers on AI quality engineering practices
Seniority
Senior, hands-on IC
