Fullstack AI Quality Engineer – Applied & Agentic AI Systems
Core
Design and implement comprehensive testing strategies, automation, and evaluation frameworks for Generative AI, LLM-based, and agentic systems to ensure reliability, fairness, and compliance in healthcare applications.
Role type
Senior Fullstack AI Quality Engineer (Applied & Agentic AI)
Builds
High-value AI solutions and services for complex business challenges in healthcare
Domain
Healthcare + Applied AI / Generative AI / Agentic Systems
Deliverable
production ML models
Required skills
Generative & Agentic AI Testing Strategy, Full-Stack Test Automation, Quality Assurance Framework & Compliance, Edge Case/Bias/Fairness Testing, Security & Performance Testing, Python, TypeScript, AI Evaluation Frameworks (RAGAS, DeepEval, Promptfoo), Agentic Workflow Validation, Observability & Feedback Loops, CI/CD Integration
Preferred skills
Cloud testing (AWS), Observability & ALM tools (Grafana, Jira, GitHub), Agentic SDLC & Engineering Excellence, Security testing methodologies
Technologies
Python, TypeScript, React, Angular, pytest, allure, Playwright, Jest, Cypress, GitHub Actions, RAGAS, DeepEval, Promptfoo, LangSmith, Arize Phoenix, AWS, Grafana, Jira, GitHub
Responsibilities
Design and implement testing strategies for LLM-based applications and agentic systems; Develop automated testing solutions for continuous evaluation of LLM applications; Create and maintain comprehensive test documentation and testing guidelines; Identify and develop test scenarios for edge cases, biases, and fairness in LLM behavior; Collaborate to test for vulnerabilities specific to LLM applications and conduct performance testing
Seniority
Senior, hands-on IC