Senior Databricks Data Engineer
Core
Designing and implementing scalable data pipelines and modern data architectures on cloud platforms to ensure data quality and performance for data scientists and architects.
Role type
Senior Databricks Data Engineer
Builds
Production data pipelines, Data Lakehouse architectures (Medallion model), and optimized data sets for analytics and ML.
Domain
Cloud Data Engineering, Big Data, Data Governance
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Apache Spark (PySpark, Spark SQL, Scala), Delta Lake, Unity Catalog, CI/CD integration, Batch and Streaming data processing, Cloud architecture design
Preferred skills
Data Quality checks, Medallion architecture design, Performance optimization
Technologies
Databricks, Azure/AWS/GCP, Apache Spark, Delta Lake, Unity Catalog, MLflow, Power BI
Responsibilities
Design and implement complex ETL/ELT data flows; Optimize Delta Lake tables for read/write performance; Configure security and access controls via Unity Catalog; Collaborate with Data Scientists on ML model integration; Implement automated workflows and data quality monitoring.
Seniority
Senior, hands-on IC
