Data Engineer (GCP, Azure)
Core
Crafting and maintaining robust data pipelines to ensure efficient extraction, transformation, and loading (ETL) of data for consulting missions.
Role type
Data Engineer
Builds
Scalable data solutions integrating cloud services and machine learning models
Domain
Data consulting and data-driven digital marketing
Deliverable
production ML models
Required skills
Python, SQL, database management, ETL pipeline development, cloud architecture, Spark, Kafka
Preferred skills
Machine learning modeling, real-time data processing
Technologies
MS Azure, GCP, AWS, Spark, Kafka
Responsibilities
Building and optimising data pipelines to facilitate seamless data flow across systems and platforms; Managing databases, ensuring their integrity, and implementing data storage and retrieval solutions; Leveraging cloud services such as MS Azure, GCP, and AWS to architect and deploy scalable data solutions; Collaborating with teams to integrate machine learning models into data pipelines for enhanced data processing; Implementing and working with Spark and Kafka for real-time data processing and analytics
Seniority
Individual Contributor
