Data Engineer
Core
Design, develop, and maintain data solutions for data generation, collection, and processing, including creating data pipelines and implementing ETL processes.
Role type
Data Engineer
Builds
Data pipelines, data assets, and offerings
Domain
Cloud data and AI
Deliverable
production ML models
Required skills
Microsoft Fabric, Azure Synapse Analytics, Azure Databricks, PySpark, Scala, SparkSQL, Purview, Unity Catalog, Azure Stream Analytics, Azure Data Explorer, Azure Time Series Insights, Python, LLM, Open AI Foundry, Github Copilot
Preferred skills
Data science, Real time analytics
Technologies
Microsoft Fabric, Azure Synapse Analytics, Azure Databricks, Azure Stream Analytics, Azure Data Explorer, Azure Time Series Insights, Python, LLM, Open AI Foundry, Github Copilot
Responsibilities
Design and develop comprehensive data solutions; Create efficient data pipelines; Implement extract, transform, and load processes; Lead Big data architecture and design of Lambda architecture; Analyze and assess the impact of requirements on data lifecycle; Handle OLTP and OLAP data workloads; Create new data assets & offerings.
Seniority
Mid-level (3-7 years IT experience, 1-3 years Big data experience)
