Data Engineer
Core
Design, build, and govern a production-grade Lakehouse data platform using Databricks, focusing on end-to-end architecture from ingestion to serving.
Role type
Senior Databricks Architect
Builds
End-to-end data infrastructure (Medallion architecture), Unity Catalog governance layer, and production-grade data pipelines.
Domain
Data Engineering / Lakehouse / Databricks
Deliverable
production ML models | infrastructure
Required skills
Databricks architecture, Delta Lake design, Unity Catalog implementation, PySpark, Delta Live Tables (DLT), data modeling, schema evolution, CI/CD integration, Spark performance optimization, cost governance, Python coding standards, unit testing frameworks
Technologies
Databricks, Unity Catalog, Delta Lake, PySpark, Delta Live Tables, Databricks Workflows, Git
Responsibilities
Design and implement Medallion Architecture (Bronze/Silver/Gold); Lead setup and rollout of Unity Catalog for data governance; Build and maintain production-grade data pipelines using PySpark and DLT; Optimize Spark execution plans and cluster configurations; Define data modeling standards and schema evolution policies; Establish data lineage tracking and access control frameworks.
Seniority
Senior, hands-on IC