CareerPlan

Senior Data Engineer

Cochin, KakkanadFull-time2026-10-29 → 2026-11-16

Core

Design, develop, test, and maintain production-grade distributed data-processing workloads using PySpark and Apache Spark, focusing on performance optimization and troubleshooting.

Role type

Senior IC data engineer (Spark internals & optimization)

Builds

Production-grade batch and incremental processing pipelines

Domain

Data engineering / Distributed systems

Deliverable

production ML models

Required skills

Python, PySpark, Apache Spark internals, SQL, performance optimization, distributed processing, data structures, exception handling

Preferred skills

Spark Structured Streaming, Kafka, Delta Lake, Apache Iceberg, Databricks, Amazon EMR

Technologies

PySpark, Apache Spark, Spark SQL, Kafka, Delta Lake, Apache Iceberg, Databricks, Amazon EMR

Responsibilities

Design and develop production-grade PySpark applications; diagnose and resolve complex performance bottlenecks; build and maintain reliable batch and incremental processing pipelines; troubleshoot job failures and performance regressions; evaluate technical trade-offs and collaborate with engineers and architects.

Seniority

Senior, hands-on IC