Data Engineer (Databricks)
Core
Migrating SQL Server solutions to a Databricks cloud platform and building scalable data engineering processes for an international insurance environment.
Role type
Senior Data Engineer (Databricks)
Builds
Scalable data transformation processes and Medallion architecture data models (Bronze/Silver/Gold) on Databricks.
Domain
Insurance / Cloud Data Engineering
Deliverable
production ML models | product features
Required skills
Python, PySpark, Databricks, Delta Lake, SQL, Clean Code, OOP, Large code repositories, Data modeling
Preferred skills
Azure Data Factory, CI/CD, Azure DevOps, Analytical tools, DevOps/Infrastructure
Technologies
Databricks, Delta Lake, Python, PySpark, SQL, Azure Data Factory, Azure DevOps
Responsibilities
Analyze and refactor complex SQL logic (stored procedures), Implement data transformations in Python/PySpark, Design and develop Medallion architecture data models, Create scalable data transformation processes, Ensure code quality, modularity, and maintainability, Participate in code reviews and architectural decisions
Seniority
Senior, hands-on IC
