Data Architect and Engineer
Core
Design and build automated data pipelines and enterprise data architecture to ingest public and internal data into governed Delta tables for analysts and business users.
Role type
Senior hands-on data architect and engineer
Builds
Automated data pipelines, governed data platforms, and shared development templates
Domain
Public sector data governance and engineering
Deliverable
production ML models
Required skills
Python, SQL, Apache Spark, Databricks, Delta Lake, Unity Catalog, REST API integration, data governance, data masking, data contracts
Preferred skills
GIS/location-based data, AI coding assistant prompt engineering, cross-agency data-sharing agreements
Technologies
Databricks, Snowflake, BigQuery, Python, SQL, Spark, Unity Catalog, Delta Lake
Responsibilities
Build and maintain automated pipelines ingesting data from public portals, APIs, and internal systems into Delta tables; Assist with platform architecture, naming standards, and documentation; Collaborate on data sharing, ownership, and quality standards; Monitor data reliability using alerts and quality checks; Prepare datasets for exploration via AI tools; Write skill files for AI coding assistants; Help departments develop pipelines using shared templates
