AI QA Engineer, AVP
Core
Design and implement scalable test frameworks to validate AI-driven product behavior, ensuring reliability, accuracy, and safety of intelligent features within enterprise investment management platforms.
Role type
Senior IC Quality Assurance Engineer (AI Testing)
Builds
Automated and manual test frameworks for AI-enabled capabilities in CRIMS/State Street Alpha
Domain
Financial services / Asset management / AI testing
Deliverable
production ML models | product features
Required skills
AI testing frameworks, probabilistic output assessment, drift detection, API testing, CI/CD integration, test strategy development, defect lifecycle management, risk-based testing, automated test execution, model behavior validation
Preferred skills
Financial services domain expertise, intelligent test generation, production monitoring setup
Technologies
CI/CD pipelines, API testing tools, test automation frameworks
Responsibilities
Develop and maintain automated and manual test frameworks for AI-enabled features; Create test plans and data strategies for functional, integration, and non-functional testing; Validate product behavior for AI use cases including output quality and exception handling; Design tests for AI guardrails and probabilistic outputs; Define approaches for monitoring AI behavior in production including drift detection; Identify and drive resolution of defects with root cause analysis; Integrate automated testing into CI/CD pipelines; Apply AI tools to improve test design and defect discovery; Collaborate on acceptance criteria and quality gates for AI user stories; Support risk-based testing prioritization; Contribute to continuous improvement of QA standards for AI products
Seniority
Mid-level to Senior, hands-on IC