QA Engineer, AI Quality and Automation - HeartStamp
Core
Stabilize and extend Playwright end-to-end test suites for AI-generated greeting cards and customer storefronts; build a structured evaluation system to score and improve AI output quality.
Role type
QA Engineer, AI Quality and Automation
Builds
Production-ready test suites and automated evaluation pipelines for AI-generated content
Domain
Generative AI / E-commerce / Print-on-demand
Deliverable
production ML models | product features
Required skills
Playwright automation, LLM evaluation rubric design, regression tracking, full-stack web testing (Next.js/React, Python/FastAPI), RAG and AI orchestration systems understanding
Preferred skills
LangChain, LangGraph, LangSmith, AI-assisted testing tools, prompt-based test generation
Technologies
Playwright, LangChain, LangGraph, LangSmith, Linear, Slack, Google Workspace, Sentry, Grafana, Hotjar, pgvector, LangGraph
Responsibilities
Audit and stabilize existing end-to-end test suites; extend coverage for AI generation and storefront surfaces; design and implement rubrics for AI output quality; translate evaluation findings into actionable items for engineering teams; produce concise async reports on test and eval status
Seniority
Mid-level (3-5 years experience), hands-on IC