Principal Data Platform Engineer

## About the Role We're looking for a Principal Data Platform Engineer to own the data platform as we migrate to a new cloud environment and modernize how data moves from applications through streams, into a governed warehouse, and out to marts that teams can actually use. This is a hands-on IC role reporting to the Senior Director of Platform Engineering. You'll be accountable for data being correct, governed, discoverable, and ready for consumption from source through curated marts. You'll partner closely with application engineering, analytics, SRE, security, finance, and executive stakeholders. ## The Stack We're on Azure and expanding across cloud providers. IaC and git-backed delivery are established. Orchestration, streaming/ingestion tooling, catalog, and table/storage formats are open decisions. You'll have real influence here. Bring depth in a modern stack and the judgment to choose well for our constraints. ## What You'll Do - Design and implement end-to-end data architecture including application CDC, streaming, schema design, transformation, warehouse/mart modeling, and consumption-readiness with IaC and automation-first delivery. - Build resilient multi-region database and replication topologies with automated failover and DR. - Architect streaming pipelines with correctness, replayability, and schema evolution as first-class concerns. - Model, document, and catalog marts for downstream self-service analytics and customer-facing consumption. - Build monitoring, alerting, and data-quality observability across data platform services. - Own PII classification, lineage, access controls, and encryption in partnership with Security. - Mentor engineers and set architectural standards across data platform craft. - Treat compute and storage spend as a design input. ## Required Qualifications - 8+ years in data engineering, platform engineering, SRE, or DevOps, with a track record at Staff or Principal IC level. - Proven experience building large-scale distributed data systems in production. - Deep hands-on cloud experience (Azure preferred; AWS/GCP transferable) with IaC and CI/CD (Terraform or comparable). - Strong SQL and data modeling fundamentals across streaming and relational/warehouse patterns. - Production depth in at least one orchestration tool and one streaming/ingestion stack. - Experience designing cross-regional database replication architectures, including failover and consistency trade-offs. - Warehouse and mart modeling with a transformation/semantic layer (dbt or comparable). - Proficiency in Python, Go, or similar for automation and tooling. - Track record of owning ambiguous, cross-team problems end to end from approach through delivery. ## Preferred Qualifications - Kubernetes for data workloads or equivalent container orchestration. - Table/lakehouse formats (Iceberg, Delta, Hudi) and judgment on where they fit against a warehouse. - Data-quality frameworks (Great Expectations, Soda) and production observability (Prometheus, Grafana, Datadog). - Schema registries and event contracts (Avro, Protobuf, JSON Schema). - Catalog and discovery tooling (Microsoft Purview, DataHub, Collibra, or similar). - GitOps tooling (Flux, ArgoCD) and FinOps practices. - Practical data governance: PII identification, lineage, encryption, and a sane access model for regulated data. - Familiarity with MCPs (Model Context Protocol) and exposing data to AI systems safely. - Background in fast-paced, high-growth environments with hard delivery deadlines. ## Technical Skills ### Cloud & Infrastructure - Cloud IaaS: Azure (primary), AWS or GCP transferable - IaC: Terraform or comparable; git-backed, automated delivery - CI/CD: GitHub Actions, GitLab CI, ArgoCD, or similar ### Data Platform & Streaming - Orchestration: Airflow, Prefect, Dagster, or similar - Streaming & ingestion: Kafka, Flink, Airbyte, or similar - CDC, event streaming, batch ingestion, schema registries, and event contracts - Warehouse: Snowflake or comparable; transformation/semantic layer (dbt or comparable); mart modeling - Catalog and table/lakehouse formats (decisions in flight) - PostgreSQL and operational RDBMS at scale; cross-region replication and DR patterns ### Governance & Security - PII classification/labeling, lineage, change management, data-access model, encryption for regulated client data - Secret management: Azure Key Vault or similar ### Programming & Observability - Python, Go, or similar; SQL; YAML/JSON/HCL for IaC and pipelines - Metrics, logging, tracing, alerting, and data-quality monitoring for pipelines and platform services ## What We Screen For Beyond the Technical Bar - Written-first communication - designs, ADRs, and runbooks live in the repo. - Ownership and bias to action - you drive ambiguous problems to working systems without waiting for the next step. - Influence without authority - you build consensus and can hold the line on governance when it matters. - Principled prioritization - you protect the critical path and can explain why you said no. - Stakeholder and executive communication - you translate complex technical reality into clear risk, cost, and timeline. - Mentorship - you level up the engineers around you through design reviews, pairing, and documentation. - Self-awareness and collaboration - you know your edges and partner well across application, analytics, SRE, and security.