Senior Data Engineer, Regulatory Systems
Core
Design, build, and maintain scalable data platforms and pipelines (Lakehouse/Warehouse) to power trusted analytics, regulatory intelligence, and AI-ready data products for policy development and compliance.
Role type
Senior IC data engineer (cloud & data platform)
Builds
Scalable data pipelines, Lakehouse/Warehouse solutions, curated datasets, and semantic models for analytics and AI.
Domain
Public sector / Government / Regulatory systems
Required skills
Microsoft Fabric, Azure, Spark/PySpark, Delta Lake, Medallion architecture, dimensional/Data Vault modelling, advanced SQL, Python, CI/CD, DevOps/DataOps
Preferred skills
Microsoft Purview, AI/ML-enabled pipelines, Agile/Scrum, Microsoft certifications
Technologies
Microsoft Fabric, Azure, Databricks, Power Platform (Dataverse), Azure DevOps, GitHub, Python, Spark, SQL
Responsibilities
Design and maintain scalable batch and near real-time data ingestion/transformation pipelines; Implement Lakehouse/Warehouse solutions using Medallion architecture; Develop curated datasets and semantic models; Implement data quality, validation, monitoring, and testing practices; Apply data governance and security controls; Contribute to CI/CD processes and environment management; Collaborate with stakeholders to translate requirements into data solutions.
Seniority
Senior, hands-on IC with mentorship