Senior/Lead Quality Engineer - Chatbot
Core
Founding Quality Engineer defining quality foundations, testing strategies, and release processes for a conversational AI chatbot product with dynamic, non-deterministic outputs.
Role type
Senior/Lead Quality Engineer (LLM/Chatbot)
Builds
Chatbot product components and agentic AI workflows
Domain
Generative AI / Conversational AI / LLM-powered applications
Deliverable
production ML models | product features
Required skills
Software quality engineering, test automation, programming proficiency, LLM/system testing, output evaluation framework design, AI risk/guardrail testing, performance testing, technical collaboration
Preferred skills
AI security/safety testing, conversational AI platform experience, first Quality Engineer in a team, mentoring engineers
Technologies
LLMs, RAG, Chatbot platforms, Test automation frameworks
Responsibilities
Develop and evolve quality practices and test strategy for the Chatbot product; Establish and run an output-evaluation framework for assessing Chatbot quality and regressions; Test dynamic and non-deterministic LLM-based systems; Identify, test, and improve guardrails for AI-specific risks; Partner with engineers through code reviews and collaborative debugging; Investigate testing gaps and research relevant methods/tools; Design proof-of-concepts for new testing approaches; Support the evolution of release testing; Contribute performance-testing thinking for LLM-powered workflows; Share knowledge and build the Quality Chapter across the engineering organization