Consultant, Data Engineer | Databricks
Core
Design and build Databricks and Delta Lake data pipelines to move ITDM data into reliable, reusable Enterprise Data Lakehouse (EDL) layers for a Fortune 100 company.
Role type
Senior Data Engineer (Consultant)
Builds
Enterprise Data Lakehouse layers, data pipelines, and reusable engineering patterns for IT demand, capacity, and cost reporting.
Domain
Insurance/Financial Services, Data Engineering, Cloud Data Platforms
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Databricks, Delta Lake, Python, SQL, Data Pipeline Design, Data Modeling, Data Quality Checks, Performance Optimization, Cloud Data Platforms
Preferred skills
Agile methodology, Stakeholder collaboration, Production support, Documentation
Technologies
Databricks, Delta Lake, Python, SQL, JIRA, Jenkins, Confluence, Git
Responsibilities
Design and build Databricks and Delta Lake data pipelines; Develop Python and SQL transformations and data models; Establish scalable lakehouse patterns for ingestion and processing; Partner with engineers and stakeholders to translate needs into technical solutions; Troubleshoot pipeline and data-quality issues; Create reusable engineering patterns and documentation.
Seniority
Senior, hands-on IC