Sr. Principal Architect - Observability
Core
Architecting and leading Workday's Observability Platform and Services to ensure resilient, secure, massively scalable distributed systems for real-time insights across HCM, Fins, and AI/ML products.
Role type
Sr. Principal Architect - Observability (Hands-on IC)
Builds
Next-generation distributed systems platforms for Observability, including Agentic experiences for incident detection, triage, and recovery.
Domain
Cloud Infrastructure (Private/Public), Distributed Systems, Observability, AI/ML Agents
Deliverable
production ML models | product features | infrastructure
Required skills
Distributed systems architecture, Observability platforms (Prometheus, Mimir, Elasticsearch, Clickhouse, VictoriaMetrics, Jaeger/Zipkin), OpenTelemetry, LLM orchestration (LangChain), Observability agents, Data collection agents, OSI stack deployment
Preferred skills
Low-latency/high-throughput data processing, Open source contributions to O11y frameworks
Technologies
Prometheus, Mimir, Elasticsearch, Clickhouse, VictoriaMetrics, Jaeger, Zipkin, OpenTelemetry, LangChain, MCP, Skills
Responsibilities
Provide architectural leadership for the Observability Platform; Drive Agentic experiences for incident detection and recovery; Ensure seamless Observability experience across logs, metrics, traces, and alerts; Drive standardization of Observability tooling; Collaborate with Infrastructure, Technology Operations & Product teams; Evangelize Observability to engineering teams.
Seniority
Sr. Principal, hands-on IC
