#111 - Data Engineer
Core
Migrate large-scale datasets and data pipelines to Snowflake, adapting existing workflows and ensuring data quality for a modernization initiative.
Role type
Senior Data Engineer (Migration & Platform Modernization)
Builds
Migrated data pipelines, orchestration workflows, and modernized data lake architectures
Domain
Cloud Data Engineering, Big Data, Data Warehousing
Required skills
Python, SQL, Snowflake, Apache Airflow, Apache Kafka, Apache Spark/PySpark, Iceberg, Parquet, AWS S3, Data Modeling, Pipeline Orchestration, Testing Strategy
Preferred skills
Snowflake migration, Airflow version upgrades, AI tools for development
Technologies
Snowflake, Apache Airflow, Apache Kafka, Apache Spark, PySpark, Iceberg, Parquet, AWS S3, Confluent, Hive, Snowpark Connect
Responsibilities
Migrate pipelines from Apache Airflow 2 to Apache Airflow 3; Migrate Confluent connections between cloud regions; Migrate pipelines from Hive to Iceberg using Snowpark Connect; Adapt existing data pipelines to Snowflake while maintaining data quality; Support workflow scheduling and monitoring in Apache Airflow; Partner with the team to execute time-sensitive data migrations
Seniority
Senior, hands-on IC