CareerPlan

AI Test Engineer - VLabs

Bengaluru, INFull-time2026-11-02 → 2026-11-19
Hybrid

Core

Validating the performance, reliability, and integrity of AI-enabled solutions (LLMs, OCR, document classification) in production environments for global tax and immigration service lines.

Role type

Senior IC AI Test Engineer (Quality Engineering)

Builds

Reusable Python-based evaluation frameworks, drift detection systems, ground truth datasets, and AI-as-Judge pipelines embedded in CI/CD.

Domain

AI/ML Quality Assurance, Tax & Immigration Services

Deliverable

production ML models

Required skills

Python, LLM evaluation (deepeval, RAGAS, promptfoo), drift detection, OCR/VLM testing, SQL, API testing (Postman/Newman), ground truth dataset creation

Preferred skills

LangChain, LlamaIndex, Azure AI Foundry, AWS Bedrock, regulated environment experience

Technologies

Python, Postman, Newman, SQL, deepeval, RAGAS, promptfoo, LangChain, LlamaIndex, Azure AI Foundry, AWS Bedrock

Responsibilities

Design adversarial test inputs for hallucination detection; Build and maintain ground truth datasets; Implement AI-as-Judge frameworks for scoring; Operate drift detection frameworks to gate releases; Test end-to-end agentic workflows; Translate evaluation results into actionable insights for stakeholders

Seniority

Senior, hands-on IC