Senior Data Engineer
Core
Design, develop, and maintain scalable data pipelines and ETL/ELT workflows for an insurance client using Databricks and Python.
Role type
Senior hands-on IC data engineer
Builds
Scalable data pipelines, ETL/ELT workflows, and AI Agent integrations
Domain
Insurance industry + Cloud data engineering
Deliverable
production ML models
Required skills
Databricks, Python, PySpark, Spark SQL, Delta Lake, SQL, Cloud platforms (AWS/Azure/GCP), REST APIs, Data warehousing, Lakehouse architecture
Preferred skills
Insurance domain experience (policy administration, claims, underwriting), AI Agent configuration, RAG workflows
Technologies
Databricks, Delta Lake, Unity Catalog, PySpark, Spark SQL, AWS, Azure, GCP
Responsibilities
Design and maintain scalable data pipelines; Build and optimize ETL/ELT workflows; Develop data transformation solutions; Implement batch and real-time data processing; Integrate data from enterprise systems and APIs; Implement data quality checks and monitoring; Support AI Agent configuration and RAG workflows; Collaborate with data architects and scientists.
Seniority
Senior, hands-on IC