Software Engineer in Test II
Core
Build and maintain evaluation suites for AI features (AI Assistant, agent orchestration), test guardrails and human approval gates, and execute automated/manual testing across web and mobile platforms to ensure quality in an AI-native environment.
Role type
Senior SDET (AI/LLM focus)
Builds
AI feature evaluation suites, agent behavior tests, and automated test coverage for AlertMedia's Risk Intelligence platform
Domain
AI/LLM engineering, Risk Intelligence, SaaS
Deliverable
production ML models | product features
Required skills
LLM evaluation techniques, agentic systems testing, API/integration/distributed systems testing, automation framework design, SQL/NoSQL, CI/CD workflows, AI-powered engineering tools
Preferred skills
mentoring, Python/JavaScript UI automation, Gherkin, Selenium/Cypress/Playwright
Technologies
Python, JavaScript, Selenium, Cypress, Playwright, Gherkin, Git, Jenkins, SQL, NoSQL, Codex, Claude, Cursor
Responsibilities
Build and maintain evaluation suites for AI features; Test guardrails and human approval gates under adversarial input; Design and maintain automated test coverage; Perform manual, exploratory, functional, and regression testing; Identify and track software defects; Execute API, integration, and performance testing; Partner with Engineering, Product, and Customer Success teams; Contribute to QA strategy and best practices; Mentor team members; Leverage AI-powered tools to improve QA workflows
Seniority
Senior, hands-on IC