Java QA Automation Engineer - Banking
Core
Build API and integration test coverage for Java/Spring routers and database workflows, design Playwright E2E tests for critical user journeys, and establish an LLM evaluation discipline with golden datasets and measurable quality gates.
Role type
Senior IC Java QA Automation Engineer (LLM & Banking)
Builds
Production test suites, CI quality gates, and LLM evaluation frameworks for banking clients.
Domain
Banking (BSFI) + Test Automation + LLM Evaluation
Deliverable
production ML models
Required skills
Java backend testing, Playwright E2E testing, API testing (REST, schema validation), SQL/PostgreSQL testing, CI/CD pipeline management, LLM output reliability tracking, LangChain4j, Langfuse, OpenTelemetry, Promptfoo, OpenAI Java SDK, Jackson, Swagger annotations
Preferred skills
Performance and benchmark testing, WebSocket testing, Security and dependency scanning, Legal-tech/fintech domain experience
Technologies
Playwright, MSW, GitHub Actions, Azure Pipelines, Jenkins, LangChain4j, Langfuse, OpenTelemetry, Promptfoo, OpenAI Java SDK, Jackson, Swagger
Responsibilities
Build API and integration test coverage for Java/Spring routers and database workflows; Design and implement Playwright E2E tests for critical user journeys; Create stable mocking and fixture strategies; Set up CI quality gates for linting, unit/integration/E2E tests, coverage, and reporting; Expand coverage to untested UI and workflow areas and prevent regressions; Build an LLM evaluation discipline with golden datasets and measurable quality gates; Track model output reliability, detect drift, and gate releases on quality/stability thresholds; Improve test maintainability, remove flakiness, and optimize runtime.