Staff Software Engineer, Big Data
Core
Design and maintain a scalable identity resolution platform that ingests, normalizes, links, and versions identity data from multiple sources using deterministic and probabilistic matching logic.
Role type
Staff Software Engineer (Big Data / Identity Resolution)
Builds
Scalable identity resolution platform, batch and streaming data pipelines, reliable APIs, and datasets.
Domain
Big Data, Identity Resolution, Data Governance, Privacy-Compliant Data Engineering
Required skills
Spark, Scala, large-scale data infrastructure, API development, data governance, data quality, metadata management, access controls, privacy-by-design, AWS services, data lakes, cloud warehouses
Preferred skills
AI workflow integration, regulatory compliance (GDPR/CCPA)
Technologies
Spark, Scala, AWS
Responsibilities
Design and maintain scalable identity resolution platform; Build pipelines to ingest, normalize, link, and version identity data; Implement deterministic and probabilistic matching logic; Expose identity data through documented APIs; Operate batch and streaming data pipelines; Establish data quality checks, schema enforcement, and access controls; Collaborate with legal and security teams on privacy and regulatory requirements; Monitor data quality, freshness, and integrity.
Seniority
Staff, hands-on IC with strategic ownership
