Analista QA - Quality & AI Agents
Core
QA Engineer specializing in testing conversational agents based on Generative AI, ensuring quality, reliability, and consistency of LLM-based agent responses.
Role type
QA Engineer (Generative AI Agents)
Builds
Conversational agents powered by Large Language Models (LLMs)
Domain
Artificial Intelligence / Generative AI / Conversational Interfaces
Deliverable
production ML models
Required skills
Quality Assurance, Test Automation, API testing, Test strategy design, Regression testing, LLM behavior validation, Hallucination detection, Guardrails validation, Test dataset creation, Conversational quality analysis
Preferred skills
Generative AI experience, Chatbot/Voicebot testing, NLP, Python automation, Customer Experience
Technologies
Large Language Models (LLMs), Test automation frameworks, APIs, Conversation analysis tools, Test datasets, AI evaluation pipelines
Responsibilities
Create test strategies for LLM-based conversational agents, Develop test scenarios for agent behavior validation, Automate tests for prompts and model responses, Identify hallucinations, inconsistencies, and inappropriate responses, Create test datasets and regression suites, Validate adherence to guardrails and response policies, Participate in quality analysis before agent deployment, Monitor response quality in production
Seniority
Mid-level, hands-on IC