Databricks Platform Expert
Core
Manage, optimize, and support the Databricks platform environment, ensuring high availability, security, and performance for data engineering and analytics workloads.
Role type
Senior Databricks Platform Expert (Data Engineering)
Builds
Production-grade data pipelines, optimized cluster configurations, and governed data lakehouse environments.
Domain
Cloud Data Engineering / Big Data
Deliverable
production ML models | infrastructure
Required skills
Apache Spark (PySpark/SQL/Scala), Databricks Administration, Unity Catalog, Delta Lake, CI/CD for Data, Cloud Security (IAM/Secrets), Cost Optimization, Cluster Management, Git Integration, Observability
Preferred skills
Azure Cloud, Scala, SQL Warehouses, Service Principals, VNet/Networking
Technologies
Databricks, Apache Spark, Delta Lake, Unity Catalog, Azure, Git, CI/CD tools, Azure Monitor
Responsibilities
Manage and configure Databricks workspaces, clusters, SQL Warehouses, and Serverless compute; Implement and govern Unity Catalog for data access and lineage; Optimize cluster policies, auto-scaling, and cost management; Write efficient Spark code for ETL and pipeline optimization; Monitor platform health, job reliability, and troubleshoot runtime issues; Enforce security best practices including RBAC, SCIM, secrets management, and audit compliance; Collaborate on cloud networking and develop automated observability dashboards.
Seniority
Senior, hands-on IC