Senior Databricks Data Engineer
Core
Design and implement scalable data pipelines and modern data architectures on cloud platforms, ensuring data quality and performance for batch and streaming workloads.
Role type
Senior Databricks Data Engineer
Builds
Scalable data pipelines, Delta Lake architectures, and optimized data layers for analytics and ML integration.
Domain
Cloud Data Engineering (Azure/AWS/GCP)
Deliverable
production ML models
Required skills
Apache Spark, Scala, Python, Cloud Architecture (Azure/AWS/GCP), Delta Lake, Unity Catalog, CI/CD, Data Quality checks
Preferred skills
Hadoop ecosystem, MLflow, Power BI integration
Technologies
Databricks, Apache Spark, Unity Catalog, MLflow, Power BI, Azure, AWS, GCP
Responsibilities
Design and implement complex ETL/ELT pipelines; Optimize Delta Lake tables and manage partitioning/indexing; Configure security and access controls via Unity Catalog; Collaborate with Data Scientists on ML model integration; Automate workflows and monitor data quality.
Seniority
Senior, hands-on IC
