Data Engineer / Software Engineer (m/f/d)
Core
Design, build, test, and deploy scalable data pipelines and big data applications, integrating diverse data sources including IoT and social media.
Role type
Data Engineer
Builds
Scalable data pipelines and big data applications
Domain
Cloud data engineering and big data processing
Deliverable
production ML models
Required skills
Python, Apache Spark, Hadoop ecosystem, PostgreSQL, ETL processes, workflow orchestration (Dagster, Apache Airflow), Git, CI/CD
Preferred skills
Docker, Kubernetes, Artificial Intelligence (AI), Data Science
Technologies
AWS, Apache Spark, Hadoop, PostgreSQL, Dagster, Apache Airflow, Docker, Kubernetes
Responsibilities
Design and build scalable data pipelines; integrate new data sources; modernize data delivery through cloud technologies; optimize performance, scalability, and reliability; ensure high code quality and production readiness.
Seniority
Mid-level (3 years experience)