Senior AI Scientist
Core
Designing adversarial evaluations and quantitative frameworks to ensure the safety, reliability, and regulatory compliance of generative AI and agentic systems at Vanguard.
Role type
Senior AI Scientist (AI Safety & Evaluation)
Builds
Quantitative evaluation frameworks, data pipelines for large-scale assessment, and actionable safety insights for production AI agents.
Domain
Financial services, AI safety, LLM evaluation, adversarial machine learning.
Deliverable
production ML models | dashboards & analysis | research
Required skills
Adversarial evaluation design, LLM behavior analysis, statistical experimental design, Python programming, data pipeline construction, hypothesis testing, model robustness assessment.
Preferred skills
Experience in regulated industries, knowledge of jailbreaks and prompt injection, mentorship of junior data scientists.
Technologies
Python, PyTorch, Hugging Face, Scikit-learn.
Responsibilities
Design and execute adversarial evaluations of generative AI systems; develop quantitative frameworks for measuring evaluation quality; build data pipelines to process large-scale evaluation outputs; engage with stakeholders to translate complex findings into recommendations; mentor junior data scientists.
Seniority
Senior, hands-on IC with mentorship responsibilities.