Data Engineer | TMB
Core
Building and maintaining data collection processes, designing complex data pipelines, and ensuring reliable data flow into a Data Lake.
Role type
Data Engineer
Builds
Data pipelines and frameworks for data collection and processing
Domain
Cloud data engineering (GCP, Hadoop, Kafka)
Deliverable
production ML models | infrastructure
Required skills
Advanced SQL, PL/SQL, Python, Linux/bash scripting, ETL processes, real-time data processing, data governance, data quality management
Preferred skills
Scala, CI/CD pipelines, automation tools, metadata management
Technologies
GCP, Cloudera Hadoop, Apache Spark, Apache Kafka, Kafka, MQ, SFTP, APIs
Responsibilities
Designing and optimizing complex data pipelines, implementing testing frameworks for data pipelines, collaborating with analysts and data scientists, ensuring robust data management and security, researching new technologies for pipeline performance
Seniority
Mid-level (3+ years experience)
