Quality Assurance (QA) - Lead - MiDAS
Core
Develop and own the evaluation framework for GenAI solutions, focusing on Faithfulness, Relevancy, and Hallucination detection using LLM-as-a-judge frameworks, while architecting hybrid test automation suites for deterministic and probabilistic testing.
Role type
Lead QA Engineer (GenAI & Automation)
Builds
Automated evaluation frameworks, dual-layered test automation suites (E2E UI, API, Performance), and CI/CD integrated quality checks.
Domain
Automotive software engineering, Generative AI, LLM evaluation
Deliverable
production ML models | product features
Required skills
Python, Pytest, Playwright, JMeter, GitHub Actions/Workflows, LLM-as-a-judge strategies, RAG system validation, probabilistic quality monitoring
Preferred skills
Langfuse, vector databases, engineering context catalogues
Responsibilities
Develop evaluation frameworks for GenAI solutions, architect hybrid test automation suites, embed automated quality checks into CI/CD pipelines, lead performance testing, define quality standards for AI-First Engineering.
Seniority
Senior, hands-on IC with leadership responsibilities