Lead Data Engineer - Google Cloud & Azure Databricks (Kansas City)
Core
Design and ship scalable batch and streaming data pipelines across Google Cloud and Azure Databricks, serving as the technical owner for critical data assets.
Role type
Senior IC lead data engineer (multi-cloud)
Builds
Production-grade data pipelines, data products, and multi-cloud data platforms
Domain
Data Engineering, Cloud Infrastructure, Big Data
Deliverable
production ML models
Required skills
Python, Java, SQL, Apache Spark, PySpark, Delta Lake, Unity Catalog, BigQuery, Dataflow, Confluent Kafka, Google Pub/Sub, Airflow, CI/CD, Dataform, dbt, Oracle, SQL Server, PostgreSQL
Preferred skills
Generative AI tooling, Data Mesh principles, Knowledge Catalog (Dataplex), Entropy Data, Immuta, Palantir, Power BI
Technologies
Google Cloud Platform, Azure Databricks, Apache Spark, Confluent Kafka, Airflow, Delta Lake, Unity Catalog, BigQuery, Dataflow, Pub/Sub
Responsibilities
Architect and build scalable data pipelines; lead engineering teams and set multi-cloud architecture standards; define data product requirements and roadmaps; enforce data engineering standards and testing frameworks; monitor and optimize production pipelines; evaluate emerging technologies for platform improvement.
Seniority
Senior, hands-on IC with leadership responsibilities