Lead, Big Data Administrator
Core
Deploying and managing large-scale distributed Hadoop and Kafka systems to ensure performance, availability, and scalability for data pipelines.
Role type
Senior Big Data Administrator (Infrastructure)
Builds
High-availability data processing environments and streaming pipelines
Domain
Big Data Infrastructure / Distributed Systems
Deliverable
infrastructure
Required skills
Hadoop ecosystem (HDFS, Yarn, Hive, Spark), Kafka, ZooKeeper, Kerberos, SSL/TLS, Prometheus, Grafana, Ansible, Bash, Python
Preferred skills
Disaster recovery strategy design, cluster optimization, data pipeline orchestration
Technologies
Hadoop, Kafka, ZooKeeper, Ranger, Atlas, Prometheus, Grafana, Ansible
Responsibilities
Deploy and manage large-scale distributed systems; Oversee installation, configuration, and maintenance of Hadoop and Kafka clusters; Implement high-availability and disaster recovery strategies; Perform performance tuning and resource allocation; Handle data pipeline orchestration; Ensure data security through encryption and access control; Troubleshoot complex cluster issues; Automate repetitive tasks using scripting.
Seniority
Senior, hands-on IC
