Data Engineer
Core
Build and maintain scalable Hadoop infrastructure and Big Data workflows/pipelines to support Data Scientists.
Role type
Senior Hadoop Infrastructure Engineer / Data Engineer
Builds
Scalable Big Data frameworks, Near-Real-Time and Batch Data Pipelines, ETL processes
Domain
Big Data, Distributed Systems, Hadoop Ecosystem
Deliverable
production ML models | infrastructure
Required skills
Hadoop cluster management, Cloudera Manager, Spark, Kafka, Hive, SQL, NoSQL, Map Reduce, ETL design, distributed data processing, data stream ingestion, API development
Preferred skills
Relational Database Administration (DBA), high-availability environment management, HDFS management, transaction recovery, database backup
Technologies
Hadoop, Cloudera Manager, Spark, Kafka, Hive, SQL, NoSQL
Responsibilities
Deploy and maintain Hadoop clusters, manage cluster nodes, design and build data pipelines, implement ETL processes, support performance and scalability requirements, provide data ingestion tools and access APIs
Seniority
Senior, hands-on IC
