Senior Software Engineer, Data Pipeline
Core
Develop reliable, high-throughput data pipelines processing tens of thousands of requests per second for a global personalization platform.
Role type
Senior Software Engineer (Data Pipeline)
Builds
Real-time data ingestion, transformation, and export services for marketing automation and analytics.
Domain
E-commerce / Personalization / Data Engineering
Deliverable
production ML models
Required skills
Python, Go, ETL pipelines, Google Cloud Platform, Kubernetes, Apache Kafka, PySpark, Apache Beam, MongoDB, Redis
Preferred skills
Experience with Flink, DataFlow, BigQuery, BigLake Table, DataProc
Technologies
Python, Go, Apache Kafka, Kubernetes, GitLab, Google Cloud Platform, DataFlow, PySpark, Apache Beam, BigQuery, BigLake Table, DataProc, Flink, MongoDB, Redis
Responsibilities
Develop data pipelines for high-volume real-time data processing; Improve throughput and reliability of data imports; Implement integrations with external data storage; Export data to Google BigQuery using DataFlows and PySpark; Run and support production services on GCP and Kubernetes; Review peer code to maintain high quality standards; Participate in on-call rotation for incident management.
Seniority
Senior, hands-on IC
