Agentic AI, Senior Software Development Engineer
Core
Design, build, and scale evaluation frameworks, tracing, and observability systems for Zillow's agentic AI experiences to ensure reliability and quality.
Role type
Senior backend engineer (Agentic AI evaluation & observability)
Builds
Production evaluation platforms, tracing systems, and observability tools for AI agents
Domain
Real estate technology + Agentic AI
Deliverable
production ML models
Required skills
Backend engineering, scalable system design, ML Evals & Observability frameworks, LLM orchestration, API design, cross-functional collaboration
Preferred skills
Advanced degree in CS/ML, offline/online evaluation pipeline design, build vs. buy decisions for platform infrastructure
Technologies
LangSmith, Braintrust, Promptfoo, LangChain, LangGraph, MCP, Databricks MLflow
Responsibilities
Design evaluation frameworks for agentic AI, build tracing and observability systems, create platform infrastructure for domain teams, partner with applied scientists and ML engineers, optimize scaling and cost management, stay current with emerging agentic AI paradigms
Seniority
Senior, hands-on IC