Data Engineer Databricks (Azure) - Recrutement international (Montréal)
Core
Design and develop data processing solutions on Databricks using Azure services to support strategic data projects for clients.
Role type
Senior Data Engineer (Databricks/Azure)
Builds
Data Lake/Lakehouse platforms, robust data pipelines, and advanced analytics solutions.
Domain
Cloud Data Engineering, Azure, Databricks
Deliverable
production ML models
Required skills
Databricks, PySpark, Delta Lake, SQL, Azure Data Factory, Azure Data Lake Storage, Git/CI/CD
Preferred skills
Snowflake, Terraform, Azure DevOps, Azure/Databricks certifications
Technologies
Databricks, Azure, PySpark, Delta Lake, SQL, Azure Data Factory, Azure Data Lake Storage, Git, CI/CD, Snowflake, Terraform, Azure DevOps
Responsibilities
Design, develop, and optimize data processing solutions on Databricks; Develop robust and scalable data pipelines using PySpark; Participate in designing and evolving Lakehouse architectures based on Databricks; Integrate, transform, and leverage data from multiple sources; Utilize Azure ecosystem services (ADF, ADLS, Fabric) to support Databricks solutions; Collaborate with architects, analysts, and clients in defining solutions; Contribute to automation, testing, and deployment of solutions; Optimize performance, costs, and reliability of processing; Participate in production deployment and continuous improvement of data platforms.