Data Science & Quantitative Analysis Expert
Core
Designing sophisticated benchmark tasks and evaluation challenges for frontier AI models to measure analytical reasoning and model performance.
Role type
Senior IC data science & quantitative analysis expert (AI evaluation)
Builds
AI evaluation benchmarks and analytical challenge tasks
Domain
Artificial Intelligence / Data Science / Quantitative Analysis
Deliverable
production ML models
Required skills
statistical modeling, hypothesis testing, data cleaning, exploratory data analysis, predictive modeling, Python programming, Jupyter Notebooks, Google Colab, Git
Preferred skills
AI evaluation, benchmark design, machine learning, large language models, statistical research methodology
Technologies
Python, pandas, NumPy, SciPy, scikit-learn, Jupyter Notebooks, Google Colab, Git
Responsibilities
Design realistic data analysis challenges mirroring real-world research workflows; Develop reproducible reference analyses using Jupyter Notebooks or Google Colab; Create benchmark tasks requiring objective comparisons between analytical techniques; Evaluate AI-generated analyses for correctness and statistical validity; Identify analytical errors and reasoning gaps; Collaborate with AI researchers to improve benchmark quality and consistency.
