Data Engineer
Core
Designing, developing, and optimizing scalable data pipelines and high-volume data processing solutions in a distributed environment.
Role type
Senior Data Engineer (Azure ecosystem)
Builds
Scalable ETL/ELT pipelines, optimized data processing workflows, and enterprise data architecture components.
Domain
Data Engineering / Cloud Data Platforms
Deliverable
production ML models | infrastructure
Required skills
PySpark, Python, SQL, Azure Data Factory, distributed data processing architecture, data modeling, performance tuning (partitioning, caching, join optimization), CI/CD automation
Preferred skills
Azure Databricks, Microsoft Azure ecosystem, Data Lake/Lakehouse architectures
Technologies
PySpark, Python, SQL, Azure Data Factory, Azure Databricks, Microsoft Azure
Responsibilities
Designing and maintaining scalable data pipelines using PySpark and Python; Creating and optimizing ETL/ELT processes for high-volume data; Processing structured and unstructured data from multiple sources; Optimizing data processing performance; Managing workflows in Azure Data Factory; Collaborating with Data Scientists, Architects, and Business Analysts; Ensuring data quality, consistency, and reliability; Troubleshooting performance issues in distributed processing environments; Contributing to enterprise data architecture development.
Seniority
Senior, hands-on IC
