Data Engineer, Ingestion & Change-Data-Capture
Core
Build a canonical customer table with full change history by ingesting and merging messy data from dozens of partners.
Role type
Data Engineer (Ingestion & CDC)
Builds
Canonical customer tables with accurate change history
Domain
Data Engineering / Customer Data Management
Required skills
Python, data modeling, slowly-changing-dimension (Type 2) merge logic, data type coercion, data validation, deduplication, upserts, null handling, idempotency, deterministic processing
Preferred skills
Data pipelines, data warehousing, handling money with Decimal
Technologies
Python
Responsibilities
Validate raw records and quarantine bad rows with clear reasons, implement Type 2 merge logic to prevent history duplication and gaps, coerce raw text into correct types while preserving meaning, ensure deterministic dedup and output using passed-in current time
