Principal Data Scientist
Core
Principal Data Scientist owning measurement strategy, causal impact analysis, and evaluation frameworks for LLM-based and agentic systems within Zillow's foundational platforms.
Role type
Senior individual-contributor Principal Data Scientist
Builds
Measurement dashboards, evaluation layers for LLM/agent systems, and causal inference frameworks for platform teams
Domain
Real Estate Technology / AI & LLM Evaluation
Deliverable
production ML models
Required skills
Causal inference, LLM evaluation, experimental design, platform measurement, data instrumentation, executive storytelling, quasi-experimental methods, metric definition
Preferred skills
Internal product measurement, agentic system evaluation, LLM-as-a-judge implementation
Technologies
Braintrust, LLM-as-a-judge, switchbacks, difference-in-difference, synthetic control, interrupted time-series
Responsibilities
Define and operationalize measurement strategy from raw events to executive dashboards; Own evaluation of LLM-based and agentic systems including LLM-as-a-judge frameworks; Design and analyze A/B tests and quasi-experimental causal inference designs; Identify leading indicators and push back on vanity metrics; Connect platform adoption to business and revenue outcomes; Partner with engineering to instrument metrics and build trusted data pipelines; Translate complex measurement realities into narratives for VP and C-level partners.
Seniority
Principal, strategy & mentorship
