Lead Software Test Automation Engineer | Test Architect - Agentic AI
Core
Architect and lead enterprise-grade test automation strategy for UI, API, and platform layers, specifically evolving capabilities to support Agentic AI/LLM systems with probabilistic outputs and hallucination detection.
Role type
Lead Test Automation Architect (Agentic AI)
Builds
Scalable automation frameworks, high-throughput automation programs, and quality dashboards for AI model evaluation.
Domain
Software Quality Engineering / Artificial Intelligence (LLMs/Agentic AI)
Deliverable
production ML models | product features | infrastructure
Required skills
Test automation architecture, API testing (contract/integration), UI automation, test data strategy, modern programming (Java/Python/TypeScript), CI/CD integration, observability for testing, reducing test flakiness, coaching engineers.
Preferred skills
Testing LLM/Agentic AI features, evaluation harnesses, benchmark datasets, semantic scoring, drift detection, consumer-driven contracts, security/performance testing integration.
Technologies
Playwright, Cypress, Selenium, REST-assured, SuperTest, Postman, Newman, pytest, GitHub Actions, Jenkins, Azure DevOps.
Responsibilities
Define multi-layer test automation strategy, establish automation standards and patterns, design testing approaches for non-deterministic AI outputs, build high-throughput automation projects, integrate automation into CI/CD with quality gates, serve as QE technical authority and mentor.
Seniority
Senior, hands-on IC with architectural ownership