Sr. Staff Software Engineer
Core
Shape and scale Pinterest's AI/ML evaluation platform into a unified, reliable system adopted company-wide to ensure production-grade quality and enable rapid iteration for GenAI and ML workloads.
Role type
Sr. Staff Software Engineer (AI Platform)
Builds
A unified, durable evaluation system for GenAI and ML workloads
Domain
Generative AI, Machine Learning, Distributed Systems
Deliverable
production ML models
Required skills
GenAI and RecSys evaluation design, distributed systems architecture, Python programming, automated metrics design, human feedback integration, model-based grading, failure analysis, technical strategy definition, cross-team alignment
Preferred skills
Experience with AutoEvals, OpenEvals, Ragas, robustness analysis, bias detection, privacy and security in GenAI
Technologies
Python, AutoEvals, OpenEvals, Ragas
Responsibilities
Identify modernization opportunities and scale the roadmap for a company-wide evaluation system; Define cross-team technical strategy and drive broad adoption of shared evaluation capabilities; Establish evaluation methodologies balancing automated metrics, expert judgment, and safety; Lead complex cross-team workstreams and resolve technical tradeoffs
Seniority
Sr. Staff, hands-on IC with strategic scope
