Senior Data Engineer, AI Systems
Core
Build, scale, and optimize Spark-based data pipelines and infrastructure to power AI-driven marketing personalization and recommendation systems.
Role type
Senior IC data engineer (AI/ML platform)
Builds
Production data pipelines, ML Data Lake, and data products for the DaVinci Personalization platform
Domain
Marketing technology / AI-driven personalization / Cloud data engineering
Deliverable
production ML models
Required skills
Apache Spark (PySpark), Python, GCP Dataproc, Kafka, Parquet, Delta Lake, CI/CD, Docker, Kubernetes, advanced query optimization
Preferred skills
Experience solving challenging scaling problems, event streaming data handling
Technologies
Apache Spark, PySpark, GCP Dataproc, Kafka, Parquet, Delta Lake, Docker, Kubernetes, GitHub Actions
Responsibilities
Build and maintain production data pipelines for content selection, send-time optimization, and frequency capping; Own and scale Spark-based batch pipelines; Build and maintain the ML Data Lake; Support ML engineers and scientists for model development and training; Identify and resolve performance bottlenecks in data infrastructure; Collaborate on platform architectural evolution; Release data products delivering business value
Seniority
Senior, hands-on IC
