Senior Data Engineer
Core
Lead the design, development, and maintenance of scalable and efficient data pipelines in Cloud to ensure data availability, quality, and governance.
Role type
Senior Data Engineer
Builds
Scalable data pipelines, ETL/ELT processes, data ingestion services, and APIs
Domain
Cloud Data Engineering
Deliverable
production ML models
Required skills
Python, API design, software engineering best practices, containers/OpenShift, relational and NoSQL databases, cloud platforms (AWS/Azure/GCP), Data Lakes/Warehouses, PySpark/Spark, Databricks, Big Data technologies
Preferred skills
Data streaming (Kafka), data governance, data visualization, financial sector experience, Java/Spring Boot, DevOps tools
Technologies
Python, PostgreSQL, SQL Server, MongoDB, Couchbase, AWS, Azure, GCP, Databricks, Spark, Hadoop, Kafka, OpenShift, Bitbucket, Jira, Jenkins, PyTest
Responsibilities
Design and develop scalable data pipelines and data services; Build and maintain robust, automated ETL/ELT processes; Integrate data from multiple sources including APIs and third-party systems; Develop APIs and data ingestion services; Apply software engineering best practices including CI/CD and testing; Optimize performance and scalability of data processes; Work with Data Science, BI, and Product teams; Design solutions with security and compliance in mind; Contribute to technical decisions and architecture; Document technical solutions and processes
Seniority
Senior, hands-on IC