Senior Data Engineer (Databricks Migration)
Core
Migrate a large-scale retail analytics platform from BigQuery to a Databricks Lakehouse architecture and build scalable data processing solutions.
Role type
Senior Data Engineer (Databricks Migration)
Builds
Scalable Lakehouse architectures, medallion data layers (Bronze/Silver/Gold), and Spark-based data pipelines for retail analytics.
Domain
Retail analytics and cloud data engineering
Deliverable
production ML models | product features
Required skills
Python, SQL, PySpark, Databricks, Delta Lake, Apache Spark, Airflow, CI/CD, distributed data processing, data warehousing, dimensional modeling
Preferred skills
GCP, AWS, Azure, retail analytics, pricing optimization, machine learning data workloads, platform modernization
Technologies
Databricks, Apache Spark, PySpark, Delta Lake, Unity Catalog, Airflow, GCP, CI/CD
Responsibilities
Design and implement scalable Lakehouse architectures, analyze and migrate ETL/ELT workloads, develop and optimize data pipelines, implement incremental processing strategies, build Spark-based solutions, maintain medallion architecture layers, implement data governance and security, troubleshoot and optimize SQL/Spark workloads, support production deployments
Seniority
Senior, hands-on IC