Senior Associate_Hadoop Developer_Advisory Corporate_Advisory_Bangalore Millenia
Core
Senior Data Engineer designing and deploying large-scale data processing pipelines using Hadoop, Spark, and Python for enterprise clients.
Role type
Senior IC Data Engineer (Big Data)
Builds
Data transformation pipelines, batch/streaming workflows, and analytics solutions
Domain
Enterprise Data Engineering / Big Data
Deliverable
production ML models | product features
Required skills
Apache Hadoop, PySpark, Python, Apache Airflow, Unix/Shell Scripting, SQL (Oracle, MySQL, Teradata, etc.), Kubernetes/on-prem platforms
Preferred skills
GCP, Kafka, Druid, Elasticsearch, Google BigQuery
Technologies
Hadoop, Spark, Airflow, Kafka, Kubernetes, GCP, SQL dialects
Responsibilities
Design and implement data workflows with Apache Airflow; develop PySpark data transformation pipelines for batch and streaming; optimize and tune Spark performance for large-scale data; maintain Spark applications; adhere to clean coding principles and SDLC; collaborate with data scientists and backend developers.
Seniority
Senior, hands-on IC
