Lead Data Engineer
Core
Lead and drive complex, high-impact data initiatives by designing scalable, metadata-driven data solutions and optimizing data pipelines.
Role type
Senior IC Lead Data Engineer
Builds
Scalable metadata-driven data services, ETL processes, and robust data solutions on Azure/Databricks
Domain
Insurance / Data Engineering
Deliverable
production ML models
Required skills
Databricks (SQL, PySpark), Azure Data Factory, Azure Data Lake Storage, Azure Functions, Airflow, dbt, Terraform, Bicep, ARM templates, CI/CD, Infrastructure as Code, Data Governance, Security Best Practices
Preferred skills
Insurance data experience, Software Engineering Best Practices
Technologies
Databricks, Azure, Airflow, dbt, Terraform, Bicep, ARM
Responsibilities
Lead development of metadata ingestion frameworks and orchestration processes; Design and implement scalable, reusable metadata-driven services; Architect and maintain robust data solutions using Databricks and Azure services; Establish and enforce best practices for data engineering, CI/CD pipelines, and DevOps; Collaborate with stakeholders to translate business needs into engineering solutions; Optimize existing data pipelines for performance and scalability; Drive adoption of data governance and security best practices; Mentor junior engineers.
Seniority
Senior, hands-on IC with leadership responsibilities