QA Developer, AI Assistant
Core
QA specialist ensuring the quality of an industrial AI assistant built into a connected worker platform for manufacturing, focusing on automation, evals, and adversarial testing.
Role type
Senior IC QA Engineer (AI/LLM)
Builds
Automated QA tools, eval suites, and guardrails for an AI assistant used by frontline workers and supervisors.
Domain
Industrial AI / Manufacturing / LLMs
Deliverable
production ML models
Required skills
Test automation, Python, TypeScript, LLM behavior analysis, Adversarial testing, CI/CD integration, Prompt engineering, Statistics, Jira
Preferred skills
Eval harness setup, AI tools for testing, Security testing, Test pyramid knowledge, B2B/Industrial platform experience
Technologies
Python, TypeScript, Claude Code, Promptfoo, AWS, Docker
Responsibilities
Automate QA for AI features and build tools for non-developers to add evals; Build and maintain eval suites based on real usage; Define scoring rubrics and use LLM-as-judge; Run regression checks in CI for prompts, tools, and model versions; Review real conversations and group failures into patterns; Contribute to adversarial testing (prompt injection, data leaking); Perform exploratory testing on new agents and tools.
Seniority
Mid-level (2-4 years), hands-on IC