Data Engineer (Medior Analyst)
Core
Design, build, and maintain scalable data infrastructure and pipelines to power analytics, products, and decision-making processes.
Role type
Medior Data Engineer
Builds
Scalable batch and streaming data pipelines
Domain
Cloud data engineering (GCP)
Deliverable
production ML models
Required skills
Batch and streaming data pipeline development, BigQuery (partitioning, clustering, cost-aware design), Cloud Composer (Airflow) workflow orchestration, Python for data processing and unit testing
Technologies
Dataflow, Pub/Sub, Cloud Storage, BigQuery, Cloud Composer (Airflow), Python
Responsibilities
Build and maintain data pipelines, perform and document unit tests for pipelines, support UAT tests for data products, coordinate data pipeline deployment, follow data architecture and governance policies, document data pipelines, train support teams on deployed pipelines
Seniority
Medior, hands-on IC
