CareerPlanSign in

Lead Software Engineer in Test

USA🌐 Remote💼 Full-time🗓 2026-07-28 → 2026-09-26

Core

Lead quality engineering strategy and execution for next-generation AI-powered applications and distributed systems, focusing on Agentic AI, LLM workflows, and autonomous agents.

Role type

Senior IC Lead Software Engineer in Test (Agentic AI)

Builds

Scalable automation frameworks, regression test suites, and validation pipelines for UI, API, backend, and AI orchestration layers.

Domain

Healthcare technology + Artificial Intelligence (Agentic AI, LLMs, RAG)

Deliverable

production ML models | product features

Required skills

Automation framework design (JavaScript/TypeScript, Java), AI agent behavior validation, API/backend/UI testing, CI/CD integration, Cloud-native environments (Azure/AWS/GCP, Docker, Kubernetes), Distributed system testing, Test environment design, Code quality & observability practices, Mentorship

Preferred skills

Azure DevOps/GitHub Actions, AI observability tools, RAG pipeline validation, Vector databases, HIPAA compliance knowledge, Event-driven architectures, Enterprise test reporting

Technologies

Playwright, Cypress, Postman, K6, LangGraph, CrewAI, AutoGen, Semantic Kernel, Azure, AWS, GCP, Docker, Kubernetes, SQL Server

Responsibilities

Design and maintain scalable automation frameworks and regression test suites; Validate complex AI agent behaviors (multi-agent coordination, tool calling, prompt chaining); Develop functional, integration, regression, and performance testing strategies; Collaborate with engineering teams to integrate testing in SDLC; Build reusable automation solutions for APIs and orchestration layers; Mentor QA engineers and drive QA standards.

Seniority

Senior, hands-on IC with leadership responsibilities

Sourced via codingjobboard · Listed on CareerPlan, which tracks 70,000+ jobs from 20+ sources.