Sr. Software Development Engineer in Test
Seniority
Sr. Software Development Engineer in Test (Sr. SDET)
Domain
AI, AI Voice Agent services, Agentic AI, business communications
Required skills
5+ years of experience in software engineering or SDET roles, Strong programming skills in Python (preferred), Java, or JavaScript, Experience testing distributed, cloud-native SaaS systems and APIs, Hands-on exposure to LLMs or AI/ML systems, Understanding of non-deterministic systems and probabilistic testing approaches, Experience building test frameworks and scalable automation systems, Familiarity with AI evaluation techniques, Experience with CI/CD pipelines, Strong collaboration skills, Bachelor's degree in Computer Science or equivalent practical experience
Preferred skills
None explicitly listed as preferred over required, though Python is noted as preferred
Technologies
Python, Java, JavaScript, OpenAI, Claude, Gemini, Jenkins, GitHub Actions, Google Cloud Platform, Cloud Run, App Engine, Kubernetes, Datastore, Redis, ElasticSearch, Vue3, React, LiveKit
Responsibilities
Own end-to-end quality for agentic features and workflows, Design and build automation tooling and frameworks for AI/LLM-driven systems, Develop and maintain evaluation frameworks to measure response quality, accuracy, and hallucination rates, Drive automation coverage using deterministic and probabilistic validation approaches, Integrate AI quality checks into CI/CD pipelines, Build tooling for LLM observability and debugging, Partner with Applied AI teams on prompt engineering and model selection, Design and execute performance and load tests for AI services, Identify and mitigate risks related to hallucinations, bias, safety, and edge cases, Define and track AI quality KPIs, Participate in design and architecture reviews, Mentor engineers and contribute to raising the bar on AI quality engineering practices
