CareerPlan

Specialist, AI Evals Engineer

IND - Telangana - Hyderabad (Hitec City Raidurg), INFull-time2026-11-16 → 2026-11-19
Hybrid

Core

Design, build, and run evaluations of AI systems across the stack (foundation models, RAG pipelines, agentic applications) to assess quality, accuracy, robustness, and safety for a global healthcare biopharma company.

Role type

Specialist AI Evals Engineer

Builds

Evaluation datasets, test suites, metrics, scoring rubrics, and reusable evaluation pipelines integrating into AI engineering delivery.

Domain

Healthcare biopharma / Generative AI

Deliverable

production ML models

Required skills

GenAI, NLP, Machine Learning, AI Model Evaluation & Testing, RAG Evaluation, Agentic AI Evaluation, AI Evaluation Frameworks & Benchmarks, LLM-as-a-Judge Methodologies, AI Safety Testing / Red Teaming, Python Programming, Evaluation Dataset Creation, Metrics & Scoring Rubrics Development, Evaluation Pipeline Development, Statistical Analysis

Preferred skills

AI Testing & Evaluation Tools, AI Security & Red Teaming Tools, AI Governance & Responsible AI Tools, Qualitative & Quantitative Research Methods, Cloud Platforms (AWS / Microsoft Azure), Pharmaceutical / Life Sciences Domain Knowledge, Bioinformatics, Statistics, Mathematics, Physics background