Data Engineer
Core
Design, build, and operate modern data solutions on Databricks to support AI/ML, analytics, and enterprise insights.
Role type
Senior Data Engineer (Databricks/Lakehouse)
Builds
Production-grade batch and streaming data pipelines, governed data assets, and data products for AI/ML.
Domain
Insurance & Health / Data Engineering & Lakehouse
Deliverable
production ML models
Required skills
Python, PySpark, SQL, Apache Spark, Delta Lake, Databricks (Jobs, Auto Loader, Structured Streaming, Unity Catalog), Workflow Orchestration, Data Modeling, Cloud Platforms (Azure/AWS/GCP), Infrastructure as Code (Terraform), REST APIs, Data Governance
Preferred skills
Snowflake, Apache Iceberg, Apache Hudi, BigQuery, AI-assisted development tools, Insurance/Health domain data
Technologies
Databricks, Delta Lake, Unity Catalog, Azure Data Lake Storage, Amazon S3, OneLake, GitHub Copilot, Claude
Responsibilities
Design and develop scalable batch and streaming data solutions; Build Databricks-native solutions using Delta Lake and Lakeflow; Develop production-grade ingestion and transformation pipelines; Implement data-quality validation, monitoring, and observability; Integrate data solutions with governance capabilities; Troubleshoot and optimize Spark workloads; Maintain workflow orchestration and technical documentation.
Seniority
Senior, hands-on IC