Senior QA Engineer, AI
Core
Lead QA strategy and evaluation for AI/LLM applications, ensuring model output accuracy, safety, and compliance.
Role type
Senior IC QA Engineer (AI/ML)
Builds
AI-powered applications with validated model outputs and robust test gates
Domain
Artificial Intelligence / Machine Learning / LLMs
Deliverable
production ML models
Required skills
AI/LLM evaluation, Python scripting, API validation, SQL, UI automation, model versioning, prompt injection testing, compliance frameworks
Preferred skills
RAG systems testing, LangSmith/Ragas/DeepEval, cloud AI platforms, fintech domain
Technologies
Python, Selenium, Playwright, Cypress, LangSmith, Ragas, DeepEval, AWS SageMaker, Azure ML, GCP Vertex AI
Responsibilities
Lead test strategy for AI/ML features including model output validation and regression testing; Build evaluation frameworks for accuracy, bias, and hallucination detection; Define golden datasets and acceptance criteria with data science; Automate prompt-based test suites into CI/CD; Mentor QA team on AI testing standards; Monitor production AI behavior and evolve test coverage.
Seniority
Senior, hands-on IC with mentorship
