Senior Data Infra Engineer
Core
Design, build, and scale the batch and streaming data infrastructure powering user feeds, recommendations, and ML models for tens of millions of users.
Role type
Senior Data Infrastructure Engineer
Builds
High-throughput Kafka streaming pipelines and large-scale Spark batch processing jobs
Domain
Internet / Local News & AI
Deliverable
production ML models
Required skills
Spark, Kafka, Python, SQL, Hadoop, Presto/Trino, Flink, AWS (S3, EMR, Glue, Redshift), data modeling, pipeline orchestration (Airflow), cost optimization
Preferred skills
Scala
Responsibilities
Design and operate high-throughput Kafka pipelines for user events; Build and maintain batch pipelines processing billions of rows daily; Drive pipeline observability with data quality checks and SLOs; Model structured and unstructured data for analytics and product use cases; Partner with data scientists and platform engineers to drive infrastructure decisions; Improve efficiency by reducing compute costs and job latency; Contribute to shared infrastructure standards and mentor junior engineers
Seniority
Senior, hands-on IC