Senior Data Engineer
Core
Build, extend, and operate a near real-time streaming data platform for operational data across a healthcare benefits business.
Role type
Senior IC Data Engineer (Streaming & Infrastructure)
Builds
A live production streaming platform ingesting CDC events from operational databases and SaaS sources into canonical, contract-validated form.
Domain
Healthcare / Data Engineering / Streaming
Required skills
Apache Kafka (production), Apache Spark (PySpark), AWS (MSK, EMR, Glue, S3, Athena), Infrastructure as Code (AWS CDK, Terraform), SQL, Python, REST APIs, Change Data Capture (CDC), Open Table Formats (Iceberg, Delta Lake), Data Governance/Cataloging
Preferred skills
Apache Flink, JVM-based languages (Kotlin, Java), AI/Agentic data consumption patterns
Technologies
Apache Kafka, Apache Spark, AWS CDK, Terraform, Datadog, CloudWatch, Debezium, Iceberg, Delta Lake, Glue Data Catalog, Unity Catalog, Atlan, Alation, Collibra
Responsibilities
Own latency/throughput SLOs, monitoring, alerting, replays, backfills, connector-failure recovery, and dead-letter triage; Build and extend streaming pipelines ingesting CDC events; Transform and enrich data using Spark (Structured Streaming or dbt-on-Spark) including cross-stream joins; Enable secure, governed data access over PHI with classification and access policies; Ensure reliability and observability with idempotent/replayable design and data-quality validation; Provision platform services on AWS using IaC with CI/CD; Partner with upstream producers and downstream consumers on data contracts and access needs.
Seniority
Senior, hands-on IC
