Data Engineer - Web Scraping (3 months contract)
Core
Design and maintain scalable web scraping pipelines to ingest, transform, and deliver data into a data lake for luxury retail brands.
Role type
Senior IC data engineer (web scraping & ETL)
Builds
Automated data ingestion pipelines and transformed datasets for analytics
Domain
Retail / Data Engineering
Required skills
Python, ETL, data modeling, SQL, NoSQL, Azure, Airflow, DBT, Terraform, Git, data security
Preferred skills
French language
Technologies
Python, Airflow, DBT, Azure, Terraform, SQL, NoSQL, Git
Responsibilities
Maintain and manage website scraping configurations; Monitor scraping configurations for errors and potential crashes; Oversee retrieved data to detect potential issues and blockages; Develop necessary pipelines to ingest data into the Datalake and perform required transformations; Coordinate with stakeholders to understand scraping task requirements and report issues; Prepare and share periodic reports on scraping activities with stakeholders.
