Senior Associate_Hadoop Developer_Advisory Corporate_Advisory_Bangalore Millenia
Core
Design, develop, and deploy large-scale data processing pipelines and workflows using Hadoop and Spark to help organizations uncover enterprise insights and drive business results.
Role type
Senior Data Engineer (Big Data)
Builds
Production data pipelines, batch/streaming processing jobs, and data assurance solutions for enterprise clients.
Domain
Big Data, Data Engineering, Cloud/On-prem Infrastructure
Deliverable
production ML models | product features | dashboards & analysis | infrastructure
Required skills
Apache Hadoop, PySpark, Python, Apache Airflow, Unix/Shell Scripting, SQL (Oracle, MySQL, Teradata, Postgres), Kubernetes/On-prem deployment
Preferred skills
Kafka, GCP, NoSQL (Druid, Elasticsearch), Google BigQuery
Technologies
Hadoop, Spark, Airflow, Kafka, Kubernetes, GCP, Oracle, MySQL, Teradata, Postgres, Unix
Responsibilities
Design and implement data workflows with Apache Airflow; develop PySpark data transformation pipelines for batch and streaming; optimize and tune performance for large-scale data processing; maintain and support Spark applications; adhere to clean coding principles and SDLC.
Seniority
Senior, hands-on IC
