Senior Data Engineer
Core
Design, develop, and orchestrate scalable data pipelines and ETL processes to support complex analytical use cases in native or hybrid cloud environments.
Role type
Senior Data Engineer
Builds
Scalable data pipelines and ETL processes for analytical use cases
Domain
Financial/Insurance sector data engineering
Deliverable
production ML models
Required skills
Python, distributed computing frameworks (Spark, Dask), relational databases (Postgres), NoSQL (MongoDB), Git, algorithms and data structures
Preferred skills
Data Science and Machine Learning libraries (PyTorch, scikit-learn), cloud PaaS/SaaS (Azure, AWS, GCP), containerization (Docker), orchestration (Kubernetes), CI-CD (Jenkins, Azure DevOps, GitHub Actions), workflow management (Dagster, Apache Airflow, Luigi)
Technologies
Spark, Dask, Postgres, MongoDB, PyTorch, scikit-learn, Azure, AWS, GCP, Docker, Kubernetes, Jenkins, Azure DevOps, GitHub Actions, Dagster, Apache Airflow, Luigi
Responsibilities
Design and develop scalable data pipelines; Design new ETL processes and refactor existing ones; Integrate solutions into cloud environments; Define specifications and technology choices with stakeholders
Seniority
Senior, hands-on IC
