Principal Engineer, Data Platforms (Architecture & Ingestion)
Core
Senior technical authority responsible for the architecture, engineering, governance, and evolution of the Unified Ingestion Platform and Federated Data Lake supporting enterprise observability, security, and operational telemetry data.
Role type
Principal Engineer, Data Platforms (Architecture & Ingestion)
Builds
A highly scalable, cloud-native data platform capable of ingesting and processing hundreds of terabytes of data daily across Azure, AWS, and GCP.
Domain
Enterprise data platforms, multi-cloud infrastructure, observability, security telemetry
Deliverable
production ML models | infrastructure
Required skills
Enterprise-scale cloud data platform architecture, Databricks platform strategy, Infrastructure-as-Code, high-scale streaming ingestion, multi-cloud governance
Preferred skills
Unity Catalog, Delta Sharing, metadata services, FinOps, Splunk rationalization, serverless computing environments
Technologies
Databricks, Delta Lake, Unity Catalog, Terraform, GitHub Actions, Kafka, Event Hub, Kinesis, Pub/Sub, NiFi, Cribl, OpenTelemetry, Open Cybersecurity Schema Framework
Responsibilities
Define and own the architecture, strategy, and multi-year roadmap for the Unified Ingestion Platform and Federated Data Lake; Architect and scale high-volume ingestion and processing capabilities supporting 250+ TB per day; Establish and govern enterprise Databricks architecture including multi-cloud account structures and workspace strategies; Lead implementation of Infrastructure-as-Code, Policy-as-Code, automated provisioning, and enterprise-grade CI/CD pipelines; Serve as the highest-level technical escalation point and provide architectural leadership across engineering, security, infrastructure, and platform teams.
Seniority
Principal, strategy & mentorship
