Senior Data Engineer
Core
Build and maintain large-scale ingestion pipelines, feature tables, and experimentation infrastructure for a real-time AdTech platform processing hundreds of millions of auction requests daily.
Role type
Senior Data Engineer (AdTech/ML-oriented data systems)
Builds
Ingestion pipelines for bid/win/impression logs, point-in-time-correct feature tables, conversion/labeling pipelines, and experimentation infrastructure
Domain
Programmatic Advertising / AdTech / Large-scale Analytics
Deliverable
production ML models | product features | infrastructure
Required skills
Expert SQL (window functions, incremental processing), Python, Spark/PySpark, ETL/ELT pipeline design, Cloud data warehousing (BigQuery), Data modeling, Event-driven/clickstream data handling
Preferred skills
GCP services (Dataflow, Pub/Sub, Beam), Streaming/near-real-time systems, Feature stores, dbt, Terraform/Kubernetes, GDPR/CCPA privacy engineering
Technologies
SQL, Python, Spark, PySpark, BigQuery, Airflow, Cloud Composer, Dagster, GCP (Dataflow, Pub/Sub, GCS, Beam), dbt, Terraform, Kubernetes
Responsibilities
Write diagnostic SQL queries, build ingestion pipelines, harmonize fields across datasets, develop point-in-time-correct feature tables, design conversion/labeling pipelines, own data serving write paths, build experimentation infrastructure, perform large-scale backfills, implement data isolation controls, develop automated data quality frameworks
Seniority
Senior, hands-on IC
