Senior Data Engineer
Core
Build and maintain large-scale ingestion pipelines, feature tables, and experimentation infrastructure for a real-time AdTech platform processing hundreds of millions of auction requests daily.
Role type
Senior Data Engineer (AdTech/ML pipelines)
Builds
Ingestion pipelines for bid/win/impression logs, point-in-time-correct feature tables, conversion/labeling pipelines, and experimentation infrastructure
Domain
AdTech / Programmatic Advertising / Large-scale Data Engineering
Deliverable
production ML models
Required skills
Expert SQL (window functions, incremental processing), Python, Spark/PySpark, ETL/ELT pipeline design, Cloud data warehousing (BigQuery), Data modeling, Event-driven/clickstream data handling
Preferred skills
GCP services (Dataflow, Pub/Sub, Beam), Streaming/near-real-time systems, Feature stores, dbt, Terraform, Kubernetes, GDPR/CCPA privacy engineering
Technologies
BigQuery, Spark, PySpark, Airflow, Cloud Composer, Dagster, GCP (Dataflow, Pub/Sub, GCS, Beam), dbt, Terraform, Kubernetes
Responsibilities
Write diagnostic SQL queries, build ingestion pipelines, harmonize fields across datasets, develop feature tables and aggregation pipelines, design conversion/labeling pipelines, own data serving write paths, build experimentation infrastructure, perform backfills, implement data isolation controls, develop data quality frameworks
Seniority
Senior, hands-on IC