Senior Databricks Data Engineer
Core
Design, develop, and optimize scalable, secure, and governed data pipelines and platforms on Databricks-on-AWS to support analytics, reporting, and AI initiatives.
Role type
Senior Databricks Data Engineer
Builds
Scalable batch and streaming data pipelines, Medallion Architecture data layers, and governed data solutions
Domain
Cloud data engineering on AWS with Databricks
Deliverable
production ML models
Required skills
Databricks Lakehouse Platform, Delta Lake, Unity Catalog, PySpark, SQL, Python, Medallion Architecture, Delta Live Tables, AWS S3/IAM, Git, CI/CD
Preferred skills
Spark performance tuning, API development, local testing with pytest, regulated-data environments
Technologies
Databricks, AWS, PySpark, Spark SQL, Delta Lake, Unity Catalog, Git
Responsibilities
Configure and optimize Databricks workspaces and AWS infrastructure; Design and build scalable batch and streaming pipelines; Implement Medallion Architecture and analytics-ready data models; Configure Unity Catalog, permissions, and data-access controls; Ingest data from third-party APIs and operational systems; Implement Git-based CI/CD workflows and automated testing
Seniority
Senior, hands-on IC