la26.042 - Data Engineer
Core
Lead complex data engineering projects and manage the delivery of scalable, cloud-based data pipelines for Fortune 100 clients using Azure Databricks and Medallion Architecture.
Role type
Senior Data Engineering Lead (Azure Databricks)
Builds
Production-grade data pipelines and Lakehouse architectures (Bronze, Silver, Gold layers)
Domain
Cloud Data Engineering / Big Data
Deliverable
production ML models | product features | infrastructure
Required skills
Azure Databricks, PySpark, Azure Data Factory, Delta Lake, Terraform, Git, Azure DevOps, Unity Catalog, Spark cluster optimization
Preferred skills
Medallion Architecture design, Enterprise data governance, CI/CD automation
Technologies
Azure Databricks, PySpark, Azure Data Factory, Delta Lake, Terraform, Git, Azure DevOps, Unity Catalog
Responsibilities
Design and optimize scalable cloud-based data pipelines; Manage Lakehouse pipeline architecture across Bronze, Silver, and Gold layers; Implement data governance frameworks and access controls; Automate infrastructure deployment using IaC and CI/CD; Monitor and tune performance of distributed Spark clusters
Seniority
Senior, hands-on IC with leadership responsibilities
