Databricks Data Engineer
Core
Design, develop, and maintain complex data pipeline and API products for business-critical operations and large-scale analytics using Spark and Delta Lake.
Role type
Senior Data Engineer (Big Data/Cloud)
Builds
Data pipelines, APIs, and data lake solutions
Domain
Big Data, Cloud Data Engineering, Analytics
Deliverable
production ML models
Required skills
Python, PySpark, SparkSQL, Databricks, Delta Lake, Airflow, dbt, SQL, Django/Flask/FastAPI, relational databases, Kafka, data modeling (Kimball/star schema, medallion architecture)
Preferred skills
Scala, Microsoft Fabric, database deployment pipelines, unit testing frameworks
Technologies
Databricks, Spark, Delta Lake, Azure, AWS, GCP, Airflow, dbt, Kafka, EventHub, Django, Flask, FastAPI
Responsibilities
Architect and implement complex data systems with minimal oversight; Design and maintain data pipelines supporting analytics; Partner with data science and engineering teams to solve data challenges; Stay current with fast-moving data engineering technologies.
Seniority
Senior, hands-on IC