Lead Data Engineer
Core
Design, build, and maintain secure, scalable ETL/ELT pipelines connecting enterprise systems and APIs to a unified data model, while modernizing legacy data warehouses into a data lakehouse platform.
Role type
Lead Data Engineer
Builds
Enterprise data pipelines, data lakehouse architecture, and semantic data models for reporting, analytics, and AI/ML engineering.
Domain
Enterprise Data, AI, and Automation
Deliverable
production ML models
Required skills
SQL, Python, PySpark, Apache Airflow, Docker, Terraform, CI/CD, Data Lakehouse Architecture, Enterprise System Integration, Data Modeling, Data Orchestration
Preferred skills
Informatica, Monte Carlo, Great Expectations, Soda, Databricks Unity Catalog, Machine Learning pipelines, NLP, Vector Database/RAG workflows, Infrastructure-as-Code
Technologies
Airflow, Azure, CI/CD, Cloud Data Warehouse, Databricks, Docker, ETL, GitHub, Informatica, Machine Learning, Python, PySpark, RAG, SQL, Terraform, Unity
Responsibilities
Design and maintain secure, scalable ETL/ELT pipelines; Develop and update data warehouse schemas; Lead the transition to a modern data lakehouse; Establish automated data validation and lineage tracking; Mentor teammates in modern data engineering practices.
Seniority
Senior, hands-on IC with leadership responsibilities