Data Engineer
Core
Design and build scalable ETL/ELT data pipelines, optimize cloud data storage, and ensure data quality and lineage for clients.
Role type
Senior Data Engineer
Builds
Scalable data pipelines and cloud data platforms
Domain
Cloud data engineering and analytics
Deliverable
production ML models | product features
Required skills
PySpark, Apache Spark, Python, SQL, ETL frameworks, data warehousing, distributed systems, data lineage, schema evolution, CDC patterns, Azure Synapse, Azure Data Lake, DevOps pipelines, monitoring and alerting
Preferred skills
French or Dutch language proficiency
Technologies
PySpark, Apache Spark, Python, SQL, Azure Synapse, Azure Data Lake, Delta Lake
Responsibilities
Design and build scalable ETL/ELT data pipelines; Optimize data storage and performance in cloud platforms; Ensure data quality, validation and lineage; Operate pipelines in production environments; Collaborate with analytics and AI teams; Support incident resolution and platform stability; Handle and oversee Python and SQL development activities in production environments; Oversee Spark / PySpark data processing workloads to ensure reliability and performance; Ensure proper handling of Lakehouse and Delta Lake concepts across data pipelines; Supervise and apply data partitioning strategies, schema evolution and CDC patterns; Coordinate and manage the use of Azure services, including Synapse, Data Lake and data pipelines; Oversee monitoring and alerting processes and support the use of DevOps pipelines to ensure operational stability.
Seniority
Mid-Senior, hands-on IC