Data Engineer- GCP/Databricks
Core
Design, develop, and optimize scalable batch and streaming data pipelines and cloud-native platforms for AI and analytics use cases.
Role type
Senior hands-on IC data engineer (GCP/Databricks)
Builds
Production-grade data pipelines (ingestion, transformation, curated delivery) and lakehouse architectures
Domain
Cloud data engineering (GCP) and Lakehouse (Databricks)
Deliverable
production ML models
Required skills
Databricks, Apache Spark, Delta Lake, Python, Scala, SQL, CI/CD, data modeling, ETL/ELT, data governance, schema evolution
Preferred skills
GCP Professional Data Engineer certification, Vertex AI, Cloud Functions, Dataproc, Docker, Kubernetes, Terraform, factory/industrial data sources
Technologies
GCP, BigQuery, GCS, Databricks, Kafka, Unity Catalog, Dataplex, Atlan, Collibra
Responsibilities
Design and build scalable batch and streaming data pipelines across bronze, silver, and gold medallion layers. Develop and maintain Databricks-based pipelines using Spark and Delta Lake for lakehouse architecture. Implement SQL/PySpark transformations for wide and semi-structured data. Collaborate with AI engineers and data scientists to build pipelines that feed ML models and analytical systems. Drive engineering best practices across CI/CD, testing, monitoring, and pipeline observability. Mentor junior data engineers and contribute to documentation, code reviews, and agile ceremonies.
Seniority
Senior, hands-on IC
