Senior Azure Data Engineer
Core
Design and develop scalable data processing workflows, pipelines, and ETL/ELT processes on Azure Databricks and Apache Spark.
Role type
Senior Azure Data Engineer
Builds
Modern data platform leveraging Azure Databricks, Apache Spark, and Data Lake
Domain
Cloud data engineering, Big Data
Deliverable
production ML models
Required skills
Azure Databricks, Apache Spark, Python, SQL, Data Lake architectures, Git, ETL/ELT optimization
Preferred skills
Hive, Apache Impala, CI/CD pipelines, financial services background
Technologies
Azure Databricks, Apache Spark, Python, SQL, Git, Hive, Apache Impala
Responsibilities
Design and develop data processing workflows and pipelines; Build and maintain scalable data pipelines for analytics and reporting; Work with Azure Databricks and Apache Spark for large-scale data transformations; Develop robust solutions using Python and SQL; Integrate data from multiple sources and ensure high data quality; Optimize performance and efficiency of ETL/ELT processes; Monitor data flows and troubleshoot issues to ensure stability
Seniority
Senior, hands-on IC
