Applications Systems Specialist I - Data Analytics Division
Core
Design and implement Databricks governance structures, develop advanced Python/PySpark solutions for data pipelines, and translate audit requirements into scalable technical designs.
Role type
Senior IC data engineering specialist (Databricks)
Builds
Production data pipelines, governed data products, and Databricks solution architecture
Domain
Public sector data analytics and governance
Deliverable
production ML models | product features | infrastructure
Required skills
Databricks, Python, PySpark, Spark SQL, data governance, infrastructure as code, testing strategies, code review
Preferred skills
Unity Catalog, data stewardship, data lifecycle management
Technologies
Databricks, Unity Catalog, Python, PySpark, Spark SQL
Responsibilities
Translate business and audit requirements into scalable technical designs and data models; Develop advanced Python, PySpark, and Spark SQL solutions for data ingestion, transformation, and delivery; Define and execute testing strategies for pipelines and governed data products; Mentor engineers on Databricks and data governance; Design and implement Databricks governance structures using Unity Catalog; Define or support data ownership, stewardship, classification, and access requirements.
