Senior Data Engineer Con II
Core
Design, build, and maintain scalable batch and real-time data platforms and pipelines powering enterprise analytics, reporting, and AI using Lakehouse architecture.
Role type
Senior hands-on IC data engineer
Builds
Scalable data pipelines, Medallion Lakehouse architectures, dimensional models, and data workloads on Microsoft Fabric
Domain
Insurance technology / Cloud data engineering
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Apache Spark, Python, SQL, Lakehouse/Medallion architecture, Dimensional modeling, Microsoft Fabric, CI/CD, IaC, Agentic AI tools
Preferred skills
Kafka/Event Streams, Data Vault/Kimball, Machine Learning/Generative AI workloads, Data governance frameworks, Azure/Microsoft Fabric certifications
Technologies
Apache Spark, Microsoft Fabric, OneLake, Delta, Parquet, Kafka, Python, SQL
Responsibilities
Design and optimize ETL/ELT pipelines for diverse enterprise sources; Architect Medallion Lakehouse patterns for analytics and AI; Implement dimensional models for reporting and BI; Build data workloads within Microsoft Fabric; Establish data quality, observability, and governance frameworks; Optimize Spark workloads for performance and cost; Develop reusable engineering frameworks and standards; Implement CI/CD and DevSecOps practices; Lead technical design reviews and mentor junior engineers; Collaborate with stakeholders to translate requirements into scalable solutions.
Seniority
Senior, hands-on IC with technical leadership