CareerPlan

Senior Specialist, AI Evals Engineer

IND - Telangana - Hyderabad (Hitec City Raidurg), INFull-time2026-11-16 → 2026-11-19
Hybrid

Core

Design, build, and execute evaluations of AI systems (foundation models, RAG, agents) to assess quality, accuracy, robustness, and safety for a global healthcare biopharma company.

Role type

Senior AI Evals Engineer (hands-on IC with technical leadership)

Builds

Scalable, reusable evaluation pipelines, tooling, and datasets integrating into AI engineering delivery

Domain

Healthcare biopharma + Generative AI / LLMs / RAG / Agentic AI

Deliverable

production ML models

Required skills

AI/ML system evaluation & testing, LLM evaluation (LLM-as-a-judge), RAG evaluation, agentic AI evaluation, AI safety testing/red teaming, Python (scientific computing stack), evaluation dataset development, metrics & scoring rubrics design, statistical analysis

Preferred skills

AI evaluation platforms (DeepEval, Ragas, LangSmith, etc.), AI security/red-teaming tools (SPLX, Mindgard, etc.), AI governance tools (Credo AI, IBM AIF360), cloud infrastructure (AWS/Azure), pharmaceutical domain knowledge

Technologies

Python, LLMs, RAG pipelines, Agentic AI frameworks, AWS, Azure

Responsibilities

Lead design and execution of AI system evaluations across the stack; define evaluation datasets, test suites, and metrics; own evaluation strategy and architecture; analyze results and communicate findings to stakeholders; partner with engineering and risk teams to set standards; provide technical leadership and mentorship to evaluation engineers; stay current with AI evaluation research and drive adoption

Seniority

Senior, hands-on IC with technical leadership