Senior Data Engineer (Azure & Databricks)
Core
Design and build scalable enterprise data systems, pipelines, and models to power SOCOTEC's digital platforms, analytics, and AI applications.
Role type
Senior Data Engineer (Azure & Databricks)
Builds
Enterprise data platform, scalable ETL/ELT pipelines, Master Data Management (MDM) platform, data models for analytics and AI.
Domain
Enterprise data architecture, cloud data platforms (Azure/Databricks), built environment consulting.
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Databricks, Apache Spark, Delta Lake, SQL, Python, enterprise-scale ETL/ELT pipeline development, data modeling, data architecture, distributed systems.
Preferred skills
Master Data Management (MDM) systems, modern cloud data stacks (AWS, Azure, GCP), ERP integration (Deltek, SAP, Oracle), data governance, data lineage, metadata management, lakehouse architectures.
Technologies
Databricks, Apache Spark, Delta Lake, Python, SQL, Azure, AWS, GCP.
Responsibilities
Design and implement scalable data pipelines for ingestion and transformation from enterprise systems (ERP, CRM, databases). Build and maintain data models supporting analytics, AI, and operational systems. Lead Master Data Management (MDM) initiatives and implement custom MDM platforms. Design enterprise-scale data pipelines for batch and near-real-time processing. Partner with AI teams to deliver high-quality datasets for machine learning applications. Define standards for data modeling, pipeline design, and data quality; implement monitoring and observability systems.
Seniority
Senior, hands-on IC with architectural responsibilities
