Data Scientist
Core
Independently test, challenge, and validate machine learning, Generative AI, and agentic AI use cases to ensure safe and sustainable adoption.
Role type
Senior IC Data Scientist (AI Model Validation)
Builds
Reusable evaluations, test harnesses, and automation for AI validation at scale
Domain
Financial Services / Generative AI & Agentic Systems
Deliverable
production ML models
Required skills
Python, SQL, Generative AI architectures, RAG, LLM evaluation, test harness design, codebase navigation, statistical analysis
Preferred skills
TypeScript, LangChain, LangGraph, OpenAI Agents SDK, transformer fine-tuning, AI coding agents, traditional ML (XGBoost, Random Forests)
Technologies
LangChain, LangGraph, OpenAI Agents SDK, Pydantic, Langfuse, HoneyHive, GitHub, CI/CD
Responsibilities
Design and execute evaluations for Generative AI and agentic systems; Build reproducible test harnesses and automation; Review data, methodologies, and model outcomes; Navigate unfamiliar codebases and reproduce evaluation pipelines; Provide evidence-based technical challenge to model developers; Share knowledge on AI validation practices and standards
Seniority
Senior, hands-on IC
