Senior Big Data Engineer
Core
Design, develop, and support data engineering solutions to process and analyze patient-level healthcare data, including mapping drug and diagnosis data to standard coding frames and loading millions of electronic medical records.
Role type
Senior Big Data Engineer
Builds
Data pipelines using ETL solutions for acquiring, processing, and loading medical data into Databricks
Domain
Healthcare / Big Data Engineering
Deliverable
production ML models | product features | infrastructure
Required skills
SQL, Python, Scala, Databricks platform, Apache Spark ecosystem, ETL design, CI/CD automation, data profiling, data integration, data warehousing
Preferred skills
AI-powered productivity tools (Copilot/Cursor), database design and modelling, algorithms & design patterns, large data volume processing (TBs)
Technologies
Databricks, Azure Cloud Services, Apache Spark, Delta, Parquet, Jenkins, GitLab
Responsibilities
Oversee and guide junior and mid-level engineers; delegate tasks effectively; own delivery of assigned workstreams; build pipelines for processing big amount of medical data; participate in analysis, system design, development, deployment and maintenance; conduct peer code reviews; create comprehensive automated unit and integration tests
Seniority
Senior, hands-on IC with mentoring responsibilities