Middle Data Engineer
Core
Build, maintain, and optimise scalable ETL/ELT pipelines on Azure Data Cloud Platform to support BI reporting, analytics, and ML/AI use cases.
Role type
Middle Data Engineer
Builds
Production-ready data solutions, data models, and automated pipelines on Azure Data Cloud Platform
Domain
Cloud data engineering on Microsoft Azure
Deliverable
production ML models | product features
Required skills
T-SQL, PySpark, Python, Azure Data Factory, Microsoft Fabric, Git, CI/CD, data modeling, data validation, pipeline monitoring
Preferred skills
Real-time/streaming data architectures, PowerShell, Apache Kafka, KQL, Power BI, AI/ML workflows, AI productivity tools
Technologies
Azure Data Lake, Microsoft Fabric, Azure Data Factory, T-SQL, PySpark, Git, PowerShell, Apache Kafka, KQL, Power BI
Responsibilities
Build and optimise scalable ETL/ELT pipelines; Develop and refine data models for BI and ML; Write efficient T-SQL and PySpark code; Implement automated testing and monitoring; Contribute to data governance and lineage tracking; Support CI/CD pipelines for data assets; Partner with stakeholders to translate requirements into data solutions; Provide input on data readiness for ML projects; Contribute to ED&I data platform evolution; Propose improvements to team processes; Support performance tuning and cost optimisation.
Seniority
Middle, hands-on IC