Lead Data Engineer
Core
Technical leader designing, architecting, and governing large-scale data solutions and pipelines across multiple cloud environments to support critical business decisions for clients.
Role type
Senior IC Lead Data Engineer (Databricks/Cloud)
Builds
Large-scale batch and streaming data pipelines, data migration projects, and reusable data engineering standards on Databricks.
Domain
Cloud Data Engineering / Big Data / AI Infrastructure
Deliverable
production ML models | infrastructure
Required skills
Data architecture design, Databricks administration, Spark Scala/Python/PySpark, CI/CD automation, DataOps principles, cloud-native component orchestration, data modeling (Data Vault/Kimball), streaming and batch solution design, cost optimization, cluster management, Unity Catalog governance.
Preferred skills
Databricks Professional certification, Azure Data Engineer Associate, Google Cloud Professional Architect, AWS Solutions Architect Professional.
Technologies
Databricks, Spark, Kafka, StreamSets, MapReduce, TensorFlow, PyTorch, AWS, Azure, GCP.
Responsibilities
Define and govern technical architecture and design documentation for complex customer engagements; lead the implementation of comprehensive big data and AI applications; establish and enforce Databricks and data engineering best practices; lead technical estimation and manage execution risks within customer proposals; mentor and guide a team of Data Engineers; architect scalable streaming and batch solutions using cloud-native components; lead data migration projects from on-premises to cloud environments.
Seniority
Senior, hands-on IC with leadership responsibilities