Data Engineering Lead
Core
Lead the architecture and development of enterprise data platforms, data pipelines, and integration solutions for Fortune 500 clients in financial, healthcare, and manufacturing sectors.
Role type
Senior IC data engineering lead (technical leadership)
Builds
Scalable batch and real-time data processing solutions, data lakes, lakehouses, and data warehouses
Domain
Enterprise data engineering, cloud data platforms, AI-driven automation
Deliverable
production ML models
Required skills
Python, SQL, Spark/PySpark, Azure Databricks, Microsoft Fabric, Azure Data Factory, Synapse Analytics, ETL/ELT, data modeling, data governance, CI/CD, GitHub, Azure DevOps
Preferred skills
Delta Lake, streaming/event-driven data processing, cloud-native foundations, AI coding assistants
Technologies
Azure Databricks, Microsoft Fabric, Azure Data Factory, Synapse Analytics, Delta Lake, GitHub, Azure DevOps
Responsibilities
Lead architecture and development of enterprise data platforms; Design scalable batch and real-time data processing solutions; Provide hands-on technical guidance across Python, SQL, Spark, and cloud data; Design and optimize ETL/ELT pipelines, data lakes, lakehouses, and data warehouses; Establish standards for data modelling, data quality, governance, security, observability, and performance; Lead technical reviews, troubleshooting, performance optimization, and production readiness; Implement engineering practices and CI/CD using GitHub/Azure DevOps; Mentor data engineers and promote reusable engineering patterns and best practices
Seniority
Senior, hands-on IC with technical leadership