Lead Data Engineer
Core
Design and operate an enterprise Data Platform on Databricks, focusing on performance tuning, governance, and AI-native automation.
Role type
Senior IC Data Engineer (Databricks)
Builds
Scalable data pipelines, Lakehouse solutions, and self-healing automation on Databricks
Domain
Data Engineering / AI Operations / Cloud Infrastructure
Deliverable
production ML models
Required skills
Databricks, PySpark, SQL, Delta Lake, Unity Catalog, Spark performance tuning, CI/CD, DevOps, Observability
Preferred skills
Databricks-native AI capabilities, Agentic frameworks, Serverless Compute, Infrastructure-as-Code
Technologies
Databricks, Delta Lake, Unity Catalog, PySpark, SQL
Responsibilities
Design scalable data pipelines and Lakehouse solutions; Tune Databricks workloads for performance and cost; Establish best practices for partitioning, clustering, and workload isolation; Design and operationalize Unity Catalog for data governance; Build monitoring and self-healing automation using Databricks-native AI; Drive CI/CD workflows for Databricks assets; Lead design reviews and mentor Data Engineers; Own Databricks vendor coordination.
Seniority
Senior, hands-on IC
