Data Engineer
Core
Design and develop scalable data platforms and pipelines for analytical and operational use cases using Azure and distributed frameworks.
Role type
Senior IC data engineer (cloud data platforms)
Builds
Cloud data platforms, data pipelines, and data processing workflows
Domain
Cloud data engineering (Azure ecosystem)
Required skills
SQL, Python, Apache Spark/pySpark, Azure Data Factory, Azure Synapse, Azure Data Lake Storage, Airflow, ETL/ELT design, dimensional modeling, data warehousing, Parquet, JSON, database performance optimization
Preferred skills
Snowflake, BigQuery, Redshift, Kafka, Spark Streaming, Delta Lake, CI/CD, Git-based workflows, agile development
Technologies
Azure, Databricks, Apache Spark, Airflow, Snowflake, BigQuery, Redshift, Kafka, Delta Lake
Responsibilities
Design and implement data processing workflows using distributed frameworks; Integrate and process data from multiple sources ensuring consistency and quality; Take ownership of pipeline reliability, monitoring, and performance; Collaborate with analysts and scientists to translate requirements into solutions; Drive improvements in data engineering practices and standards; Evaluate and introduce new tools within the data engineering ecosystem
Seniority
Senior, hands-on IC
