CareerPlan

AI Test Engineer - VLabs

Bangalore, IndiaFull-time2026-11-02 → 2026-11-19
Hybrid

Core

Validating the performance, reliability, and integrity of AI-enabled solutions (LLMs, OCR, document classification, agentic workflows) in production environments to ensure they meet business performance thresholds.

Role type

Senior IC AI Test Engineer (Quality Engineering)

Builds

Reusable Python-based evaluation frameworks, drift detection systems, ground truth datasets, and automated test pipelines embedded in CI/CD.

Domain

AI/ML Quality Assurance, Document AI, LLM Evaluation

Deliverable

production ML models

Required skills

Python, LLM evaluation tools (deepeval, RAGAS, promptfoo), drift detection, SQL, API testing, ground truth dataset creation, agentic workflow testing

Preferred skills

Cloud AI platforms (Azure AI Foundry, AWS Bedrock), VLM testing, regulated environment experience

Technologies

Python, Postman/Newman, SQL, LangChain, LlamaIndex, Azure AI Foundry, AWS Bedrock, deepeval, RAGAS, promptfoo

Responsibilities

Design adversarial test inputs to expose hallucination and failure modes; Build and maintain ground truth datasets; Implement AI-as-Judge frameworks for scoring and calibration; Operate drift detection frameworks to identify performance degradation; Test end-to-end agentic workflows and API endpoints; Translate evaluation results into actionable insights for stakeholders.

Seniority

Senior, hands-on IC