Senior Data Engineer
Core
Design, build, and maintain scalable data pipelines, integrations, and Lakehouse environments to support analytics, AI, BI, and automation workloads.
Role type
Senior Data Engineer
Builds
Scalable batch and real-time data ingestion pipelines, structured/semi-structured datasets within Lakehouse environments, and automation solutions.
Domain
Agriculture / Data Engineering
Deliverable
production ML models
Required skills
Python, SQL, PySpark, Databricks, Microsoft Fabric, Azure Logic Apps, Azure Data Factory, Azure Functions, CI/CD, DevOps, DataOps, GitHub, data governance, data quality, data security, API integration
Preferred skills
Medallion architecture experience, cloud services integration, stakeholder engagement
Technologies
Python, PySpark, SQL, Databricks, Microsoft Fabric, Azure Logic Apps, Azure Data Factory, Azure Functions, GitHub
Responsibilities
Build and maintain scalable batch and real-time data pipelines; Develop and optimise Lakehouse, warehouse and integration environments; Implement data quality, governance and security controls; Support automation solutions and reusable engineering components; Collaborate with stakeholders to deliver trusted data products
Seniority
Senior, hands-on IC