Data Engineer
Core
Design, build, and maintain scalable data pipelines and ETL/ELT processes to power enterprise analytics and business intelligence on Azure.
Role type
Senior IC data engineer (cloud data platforms)
Builds
Scalable data pipelines, ETL/ELT processes, and unified data platforms on Microsoft Azure
Domain
Cloud data engineering, Microsoft Azure ecosystem
Deliverable
production ML models
Required skills
Azure Databricks, Apache Spark, Python, SQL, ETL/ELT pipeline design, Change Data Capture (CDC), Azure Data Factory, Azure Synapse Analytics, Delta Lake, Azure Data Lake Storage (ADLS Gen2), data modeling
Preferred skills
Microsoft Fabric, Power BI integration, observability tools (Azure Monitor)
Technologies
Azure Databricks, Apache Spark, Azure Data Factory, Azure SQL CDC, Debezium, Delta Live Tables, Azure Synapse Analytics, Azure Monitor, ADLS Gen2, Microsoft Fabric, Power BI
Responsibilities
Design and build scalable data pipelines using Azure Databricks and Apache Spark; Develop and optimize ETL/ELT processes; Implement Change Data Capture (CDC) solutions; Write clean, efficient Python code for data processing; Collaborate with data architects and analysts to translate business requirements; Monitor, troubleshoot, and optimize data pipeline performance; Ensure data quality, consistency, and integrity across storage layers; Support integration of structured and unstructured data sources; Participate in code reviews and promote best practices; Document data pipelines and architecture.
Seniority
Mid-Senior, hands-on IC