Senior Data Engineer
Core
Design, build, and own scalable data products and pipelines using Microsoft Fabric to deliver trusted data for digital products and analytics.
Role type
Senior IC data engineer
Builds
Scalable data pipelines, ingestion/transformation/storage patterns, and unified data models for digital products and analytics
Domain
Insurance / Data Engineering
Deliverable
production ML models
Required skills
PySpark, Python, SQL, Apache Spark, Kafka, Flink, dbt, Microsoft Fabric, Docker, Kubernetes, data modeling, data architecture, CI/CD, observability
Preferred skills
Azure, AWS, Microsoft Fabric components (Lakehouse, Warehouse, Data Pipelines, Dataflows Gen2, Semantic Models), Jenkins, GitHub Actions, Azure DevOps, generative AI tools
Technologies
Microsoft Fabric, PySpark, Python, SQL, Apache Spark, Kafka, Flink, dbt, Docker, Kubernetes, Datadog, Azure, AWS, Jenkins, GitHub Actions, Azure DevOps
Responsibilities
Design and develop scalable data pipelines and processing systems; Build reusable ingestion, transformation, and storage patterns; Integrate data from diverse sources including APIs, databases, and event streams; Implement data models and schemas for unified business views; Apply modern engineering practices including version control, testing, and automated deployments; Ensure data quality, integrity, and reliability through validation and monitoring; Optimize data workflows for performance and cost efficiency; Collaborate with product managers, engineers, and analysts to understand data requirements; Contribute to data governance practices and ensure compliance with privacy and security requirements; Mentor less experienced engineers and share knowledge
Seniority
Senior, hands-on IC