Data Engineer
Core
Designing and maintaining a scalable Medallion Lakehouse architecture and building automated data pipelines for a major Defence program's enterprise data platform.
Role type
Senior Data Engineer (Azure/Databricks)
Builds
Enterprise data platform integrating engineering, supply-chain, and corporate data
Domain
Defence / Engineering / Supply Chain
Deliverable
production ML models | infrastructure
Required skills
PySpark, Spark SQL, Azure Databricks, ETL/ELT, enterprise data modelling, data governance, data lineage, performance optimization
Preferred skills
Experience with Unity Catalog, Microsoft Purview, or Collibra
Technologies
Microsoft Azure, Azure Databricks, PySpark, Spark SQL
Responsibilities
Designing Medallion Lakehouse architecture, building automated data pipelines, developing data cataloguing and governance capabilities, integrating data across engineering and ERP systems, establishing data security and access controls, monitoring and optimizing Databricks performance
Seniority
Senior, hands-on IC
