Data Engineer
Core
Design and deliver production data platforms, pipelines, and models from requirements to support, working directly with clients.
Role type
Senior Data Engineer
Builds
Production data platforms, ETL/ELT pipelines, data models (dimensional, Medallion), and data quality monitoring
Domain
Cloud Data Engineering (Azure)
Deliverable
production ML models
Required skills
SQL, Python, PySpark, Azure Databricks, Azure Data Factory, Delta Lake, CI/CD, Git, Data Modeling, Data Quality, Performance Tuning
Preferred skills
Streaming (Spark Structured Streaming, Kafka), Governance tools (Unity Catalog, Azure Purview), Infrastructure as Code (Terraform), Power BI
Technologies
Azure Databricks, Microsoft Fabric, Azure Data Factory, Delta Lake, PySpark, SQL, Python, Git, Terraform, Kafka, Unity Catalog, Azure Purview, Power BI
Responsibilities
Design and build scalable ETL/ELT pipelines; Implement batch, incremental/CDC, and streaming loads; Build data quality checks and monitoring; Automate testing and deployment with CI/CD; Work with client stakeholders to gather requirements; Integrate data from APIs and external systems; Resolve complex production issues.
Seniority
Senior, hands-on IC