Data Engineer (Databricks)
Core
Design, develop, and maintain scalable data pipelines and data processing solutions using Databricks and PySpark.
Role type
Data Engineer (Databricks)
Builds
ETL/ELT pipelines, data ingestion and transformation workflows
Domain
Data Engineering, Cloud Data Platforms
Required skills
Databricks, PySpark, Python, SQL, Apache Spark, Delta Lake, Medallion Architecture, ETL/ELT pipeline development, data warehousing, data modeling
Responsibilities
Develop and maintain ETL/ELT pipelines using Databricks and PySpark; Work on Apache Spark, Delta Lake, and Lakehouse architecture; Build data ingestion and transformation workflows; Optimize Spark jobs and SQL queries for better performance; Implement incremental data processing and CDC; Work with cloud platforms such as Azure, AWS, or GCP; Ensure data quality, reliability, and scalability.