Databricks Tech Lead
Core
Design, build, and scale a next-generation data ingestion and processing framework on a Lakehouse platform.
Role type
Senior Databricks Tech Lead
Builds
Reusable, highly scalable data ingestion and transformation pipelines
Domain
Data Engineering / Cloud Data Platforms
Deliverable
production ML models | product features
Required skills
PySpark, Databricks SQL, data ingestion framework design, Spark job optimization, CI/CD implementation, Unity Catalog integration, data modeling, software engineering principles
Preferred skills
Kafka, dbt, Delta Live Tables (DLT), Databricks certification
Technologies
Databricks, Delta Lake, Spark, Unity Catalog, AWS, Azure, GCP, S3, ADLS
Responsibilities
Design metadata-driven data ingestion frameworks (batch, streaming, CDC); Define architectural standards for Medallion architecture; Lead technical design and mentor engineers; Optimize Spark jobs for performance and cost; Implement CI/CD pipelines; Integrate pipelines with Unity Catalog for governance
