Member of Technical Staff - Data Platform
Core
Design, build, and operate distributed systems powering X's data movement and compute, processing trillions of events daily to support product and ML workloads.
Role type
Senior IC distributed systems engineer (data platform)
Builds
High-throughput, low-latency data ingestion, transport, and processing infrastructure
Domain
AI/ML infrastructure, large-scale data platforms
Deliverable
infrastructure
Required skills
distributed systems, stream processing, Rust, Go, Scala, Kafka, Flink, Spark, Trino, Hadoop, debugging, profiling, performance optimization
Preferred skills
minimal guardrails, fast-moving environments
Technologies
Apache Kafka, HDFS, Spark, Flink, Trino
Responsibilities
Design and implement high-throughput, low-latency data ingestion and transport systems; Scale and optimize multi-tenant Kafka infrastructure; Extend and tune Spark, Flink, and Trino for demanding production pipelines; Build interfaces, APIs, and pipelines for petabyte-scale data movement; Debug and optimize distributed systems for reliability and performance; Collaborate with ML, product, and infrastructure teams to unblock critical data workflows
Seniority
Senior, hands-on IC
