Senior/Lead Data Engineer
Core
Design and deliver a modern Azure-native data infrastructure for a global mining organization, ingesting data from operational sources (APIs, IoT, ERP) to create analytics-ready datasets.
Role type
Senior/Lead Data Engineer
Builds
Robust data pipelines, lakehouse architecture (raw, curated, serving layers), and scalable data flows within Azure Synapse and Databricks.
Domain
Mining industry + Cloud Data Engineering (Azure)
Deliverable
production ML models | product features
Required skills
Azure Data Factory, Azure Synapse Analytics, Databricks (PySpark/Spark SQL), SQL (T-SQL/Spark SQL), Python, REST/SOAP API integration, CI/CD (Azure DevOps), Infrastructure-as-Code (Terraform/Bicep), Data Modeling (Star/Snowflake/Medallion), Data Governance
Preferred skills
Azure Purview, Streaming data patterns (Event Hubs/Kafka), dbt, OT data environments, Power Platform
Technologies
Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage Gen2, Databricks, PySpark, Spark SQL, Azure DevOps, Terraform, Bicep, REST, SOAP
Responsibilities
Design and build scalable data pipelines using Azure Data Factory; Develop and optimize data transformation logic in Azure Synapse Analytics; Build and maintain Databricks notebooks and jobs for large-scale processing; Implement robust API extraction patterns for REST and SOAP endpoints; Design and maintain lakehouse architecture; Ensure data quality through validation and monitoring frameworks; Collaborate with Analytics Engineers to model data for downstream requirements; Apply best practices for CI/CD and environment management.
Seniority
Senior/Lead, hands-on IC with mentorship responsibilities