Staff AI Observability & Telemetry Engineer
Core
Architect the high-fidelity perception layer for a massive-scale AI-native cloud platform to capture, store, and analyze millions of hardware/software signals per second for GPU workloads and network fabric.
Role type
Staff AI Observability & Telemetry Engineer
Builds
High-cardinality telemetry infrastructure, eBPF-based diagnostic tools, automated alerting pipelines, and metric pipelines for multi-tenant billing.
Domain
AI Infrastructure / High-Performance Computing / Cloud Observability
Deliverable
production ML models
Required skills
Go, Kubernetes metric exporters, eBPF, time-series databases, Linux performance tuning, GPU hardware metrics, network telemetry, technical leadership
Preferred skills
High-velocity engineering environments
Technologies
VictoriaMetrics, Thanos, Mimir, NVIDIA DCGM, IPMI, Redfish, Kubernetes, eBPF, BCC
Responsibilities
Architect and scale high-cardinality telemetry infrastructure; Integrate hardware-level exporters into Kubernetes observability stack; Build eBPF-based diagnostic tools for network and I/O tracing; Develop automated dashboards and alerting pipelines; Design metric pipelines for multi-tenant billing; Collaborate on observability standards for AI-native workloads; Lead technical design reviews and mentor team members.
Seniority
Staff, hands-on IC with technical leadership