Senior Data Engineer
Core
Design, develop, and deploy high-performance data pipelines (streaming, micro-batch, batch) to build aggregate and enriched data sets for business analysis and visualization.
Role type
Senior IC data engineer
Builds
Data pipelines, data lakes, and data warehouses
Domain
Cloud data engineering
Required skills
SQL, Python, data lake/warehouse architecture, query tuning, workflow management (Airflow/Oozie), data format handling (JSON, Parquet, ORC, XML), data pipeline maintenance, data policy implementation (retention, privacy, security, compliance), mentoring
Preferred skills
Real-time data ingestion (Kafka, Spark, Storm), NoSQL, data visualization tools (Looker, Tableau, PowerBI)
Technologies
Google Cloud, BigQuery, GCS, Pub/Sub, Data Flow, Kafka, Spark, Airflow, Oozie
Responsibilities
Own and drive data integration architecture; Design, develop, and deploy high-performance data pipelines; Build aggregate and enriched data sets; Document, maintain, and support data pipelines; Measure and monitor data quality and availability; Implement data policies; Mentor junior data engineers
Seniority
Senior, hands-on IC
