Platform Engineer with Cloud Expertise
Core
Administer and govern a shared Databricks platform supporting analytics and AI teams, focusing on security, cost optimization, and data standards.
Role type
Senior Platform Engineer (Databricks)
Builds
Shared Databricks platform, ingestion pipelines, and data governance frameworks
Domain
Data Engineering / Cloud Platform
Deliverable
production ML models | infrastructure
Required skills
Databricks administration, Unity Catalog, Delta Lake, Spark, Python, PySpark, SQL, RBAC, ABAC, cost optimization, CI/CD
Preferred skills
Airflow, DLT, Databricks Workflows, automated testing frameworks, observability, AI/ML methodologies
Technologies
Databricks, Unity Catalog, Delta Lake, Spark, Python, PySpark, Airflow, CI/CD, AWS, Azure, GitHub
Responsibilities
Oversee administration of Databricks workspaces, clusters, jobs, and platform guidelines; Implement and maintain RBAC and ABAC for secure data access; Define and uphold data ingestion standards, naming conventions, and Delta Lake design methodologies; Manage permissions for workspaces and catalogs including row/column-level policies; Set cost thresholds, alerts, and usage dashboards to manage expenses; Ensure platform reliability through automated testing frameworks and CI/CD templates.
Seniority
Senior, hands-on IC