Databricks Data Engineering Manager (AWS/AZURE)
Core
Design and lead scalable, secure, and high-performance cloud data platforms using Databricks on AWS and/or Azure.
Role type
Senior Databricks Data Engineering Manager
Builds
Enterprise-scale data platforms, Data Lakes, Lakehouses, and Data Warehouses
Domain
Cloud Data Engineering (AWS/Azure)
Deliverable
production ML models
Required skills
Databricks, Apache Spark, AWS/Azure, Medallion Architecture, ETL/ELT pipelines, SQL, Python/PySpark, Kafka, Airflow, dbt, Data Governance, CI/CD, DevOps, Team Management
Preferred skills
PMP/PMI-ACP/CSM/SAFe, AWS/Azure certifications, Databricks certifications, Snowflake, Data Mesh, MLOps, GenAI data engineering
Technologies
Databricks, Apache Spark, AWS, Azure, Kafka, Airflow, dbt, Snowflake
Responsibilities
Lead and manage Data Engineering teams; Drive end-to-end delivery of modern data engineering solutions; Define and implement scalable data engineering strategies and architecture standards; Oversee development of robust ETL/ELT pipelines; Ensure data platform scalability, reliability, performance optimization and cost efficiency; Drive engineering excellence through code reviews, CI/CD automation, and DevOps practices; Establish data governance, data quality, security and compliance standards; Mentor and develop Data Engineers and Technical Leads; Drive cloud data modernization initiatives.
Seniority
Senior, hands-on IC with management responsibilities