Sr. Staff Software Engineer, AI Platform
Core
Shape and scale Pinterest's AI/ML evaluation platform into a unified, reliable system adopted company-wide to ensure production-grade quality and enable rapid iteration for GenAI and ML workloads.
Role type
Senior Staff Software Engineer (AI Platform)
Builds
A unified, durable evaluation system for GenAI and ML workloads used by hundreds of engineers across Pinterest.
Domain
Generative AI, Machine Learning, Distributed Systems, Evaluation Infrastructure
Deliverable
production ML models
Required skills
GenAI and complex ML system evaluation design, distributed systems principles, Python programming, GenAI/RecSys evaluation libraries (AutoEvals, OpenEvals, Ragas), scalable platform architecture, cross-team technical strategy, automated metrics design, human feedback integration, failure analysis, GenAI risk assessment (robustness, bias, privacy, security)
Preferred skills
None stated
Technologies
Python, AutoEvals, OpenEvals, Ragas
Responsibilities
Identify modernization opportunities and scale the roadmap for a company-wide evaluation system; Define cross-team technical strategy and drive broad adoption of shared evaluation capabilities; Make durable architectural decisions to improve user experience and reliability; Establish evaluation methodologies balancing automated metrics, expert judgment, and safety; Lead complex cross-team workstreams and resolve technical tradeoffs.
Seniority
Senior Staff, hands-on IC with strategic scope
