Lead Data Engineer (w/m/d) - infas 360, Bonn infas 360
Core
Designing sustainable data architecture and implementing a performant, stable data infrastructure on Google Cloud Platform to scale data operations.
Role type
Lead Data Engineer
Builds
Standardized and integrated Geo-, Sach-, and Market-data products for clients in marketing, sales, logistics, construction, and environmental planning.
Domain
Location Data & Analytics / Spatial Data
Deliverable
production ML models | infrastructure
Required skills
Data architecture design, Cloud platform scaling, ETL/ELT processes, Python development (Pandas, NumPy, PySpark), Infrastructure as Code, Workflow orchestration (Airflow), Data quality testing (DataKitchen), Spatial SQL, VCS/CI/CD
Preferred skills
Geodata and GIS project experience
Technologies
Google Cloud Platform, BigQuery, PostgreSQL, PostGIS, Airflow, DataKitchen, Python, PySpark, Git, GitLab, GitHub
Responsibilities
Migrate on-premise PostgreSQL solutions to a cloud-based BigQuery solution; Design, implement, and optimize data pipelines for structured and unstructured data; Ensure data availability and quality through modeling best practices; Develop automated DataOps solutions for data integrity and transparency; Configure and manage Airflow for workflow orchestration and DataKitchen for testing and alerts; Utilize Spatial SQL for processing and analyzing Geo-, Sach-, and Market data; Collaborate with cross-functional teams to develop tailored data products.
Seniority
Senior, hands-on IC with leadership responsibilities
