Distributed Systems Engineer - Data Platform - Logs and Audit Logs
Core
Design, build, and operate a robust logging platform that delivers comprehensive audit logs and metrics securely to customer destinations and third-party integrations.
Role type
Distributed Systems Engineer (Data Platform)
Builds
High-throughput logging and audit log delivery pipelines, data connectors, and alerting mechanisms for customer-facing products.
Domain
Internet infrastructure, data observability, and security compliance
Deliverable
production ML models | product features | dashboards & analysis | infrastructure
Required skills
Go, distributed systems, SQL, high-cardinality metrics handling, data streaming (Kafka, Flink), observability stacks (Prometheus, Grafana), system scalability, concurrency
Preferred skills
Experience with logging platforms or SIEMs (Splunk), data streaming technologies
Technologies
Go, ClickHouse, Prometheus, Grafana, Kafka, Flink, SQL
Responsibilities
Design and build a robust logging platform ensuring reliable data transfer to customer destinations; Develop and maintain high-performance data connectors for log-shipping products; Create and manage systems for handling comprehensive audit logs adhering to compliance standards; Scale and optimize the data delivery pipeline to handle massive volumes with low latency; Collaborate with Product and engineering teams to define requirements for new logging platforms; Maintain operational health of the log delivery platform through monitoring and on-call rotation; Collaborate on the architectural evolution of the data egress platform.
Seniority
Mid-Senior, hands-on IC