Data Product Development Data Engineer | NV1+
Core
Build and maintain scalable data engineering solutions, transforming raw data into structured, reusable datasets and integrated data products within a secure enterprise platform.
Role type
Senior Data Engineer (Data Product Development)
Builds
Reusable data products, distributed data processing solutions, and integration workflows
Domain
Federal Government / Enterprise Data Engineering
Required skills
Apache Spark, Python, Azure DevOps, Apache NiFi, Apache Iceberg, data transformation, data integration, distributed processing, troubleshooting
Responsibilities
Build and maintain data engineering solutions within a large enterprise data platform; Transform and conform raw data into structured, reusable datasets; Integrate multiple datasets to develop data products; Develop distributed data processing solutions using Apache Spark and Python; Build and maintain integration workflows using Apache NiFi; Work with Apache Iceberg to support modern data storage and processing; Use Azure DevOps for source control and engineering delivery; Develop and optimise transformation and integration logic; Troubleshoot and enhance existing data engineering solutions; Work with platform, engineering and technical stakeholders to turn data requirements into working solutions