Senior Data Engineer (Pipelines & Data Quality) - Data Platform
Core
Own the pipeline-standardization and data-quality program for the enterprise lakehouse, shipping compliance gates and building trusted dashboards to reduce data incidents.
Role type
Senior IC data engineer (pipeline & data quality)
Builds
Compliance gates, data-quality frameworks, dashboards, migration playbooks
Domain
Retail data estate, enterprise lakehouse
Deliverable
production ML models | product features | dashboards & analysis
Required skills
PySpark, SQL, data-quality frameworks (Great Expectations, dbt, Soda), medallion architecture, Git/CI, cloud platforms (Azure/AWS/GCP), data modeling
Preferred skills
Streaming (Spark Structured Streaming, Flink, Kafka), Data Vault 2.0, Unity Catalog, Power BI optimization
Technologies
Databricks, Snowflake, BigQuery, Great Expectations, dbt, Power BI, Azure, AWS, GCP, Kafka
Responsibilities
Design and operate pipeline-compliance checkers; deploy data-quality frameworks and alert routing; build and publish real-time data quality dashboards; establish source change management agreements; lead legacy pipeline migration and mentor engineers; drive data-incident reduction through prevention; partner on event stream schemas; author runbooks and conduct code reviews
Seniority
Senior, hands-on IC
