Senior Associate
Core
Design, develop, and optimize data pipelines and ETL processes on Azure cloud platform to extract, transform, and load large volumes of data for clients.
Role type
Senior Associate Data Engineer
Builds
Data pipelines, ETL processes, data models, and reporting dashboards on Azure
Domain
Data Engineering / Cloud (Azure)
Deliverable
production ML models | infrastructure
Required skills
PySpark, Scala, Python, Azure Databricks, Azure Data Lake Storage, Azure Synapse Analytics, SQL, Data Warehousing, Dimensional Modeling
Preferred skills
Hadoop, Hive, HBase, CI/CD practices
Technologies
PySpark, Scala, Python, Azure, Azure Databricks, Azure Data Lake Storage, Azure Synapse Analytics, SQL Server, PostgreSQL, MySQL, Hadoop, Hive, HBase
Responsibilities
Design and optimize data pipelines and ETL processes; Implement data ingestion and storage solutions on Azure; Develop and maintain data models and schemas; Monitor pipeline performance and troubleshoot issues; Implement data security and compliance measures.
Seniority
Senior, hands-on IC