GenAI Test Automation Lead
Required skills
10+ years QA experience, End to end test architecture / test integration for GenAI development project, GenAI Experience (testing and evaluating GenAI models, prompt engineering frameworks, and RAG systems), Proficiency in Python and AI evaluation tools like LangChain, Hands-on Automation Experience in python, Hands-on experience with LLMs, Integration of multiple tools with Automation (not just selenium), Hands-on DB experience - SQL/Snowflake, CI/CD with test automation
Preferred skills
Ability to design scalable testing frameworks within cloud environments (AWS, Azure, GCP), Deep knowledge of Software Testing Life Cycle (STLC), CI/CD pipelines, and API testing
Technologies
Python, LangChain, Selenium, GitHub Actions, GitLab CI/CD, Jenkins, SQL, Snowflake, AWS, Azure, GCP
Responsibilities
Lead the design, implementation, and maintenance of scalable end-to-end test automation frameworks tailored for GenAI applications and Large Language Models (LLMs), Build automation solutions that go beyond standard UI tools, Actively test and evaluate LLMs, prompt engineering frameworks, and Retrieval-Augmented Generation (RAG) systems, Embed automated GenAI evaluation and regression suites into modern CI/CD pipelines, Perform deep data validation and backend testing by writing complex SQL queries and managing test data across cloud data platforms, Guide and upskill traditional QA automation engineers in Python, AI evaluation techniques, and the nuances of testing cognitive applications
Seniority
Lead
Domain
GenAI, LLMs, RAG systems, Test Automation
