CareerPlan

Principal Data Processing Engineer-OSS

Hyderabad, TelenganaRemoteFull-time2025-12-11 → 2026-11-16
Hybrid

Core

Principal Data Processing Engineer leading the design, implementation, and optimization of open-source data processing platforms (Apache Gluten, Velox, Spark, Flink) for a high-performance data engine.

Role type

Principal IC Data Processing Engineer (Open Source)

Builds

High-performance data processing engine components and open-source platform integrations

Domain

Distributed systems, data processing, compilers, C++/Java systems programming

Deliverable

production ML models

Required skills

C++, Java, Apache Spark, Apache Flink, Linux, performance optimization, distributed systems architecture

Preferred skills

Apache Gluten, Velox, Presto/Trino, hardware-accelerated computing

Technologies

Apache Gluten, Velox, Apache Spark, Apache Flink, C++, Java, Linux

Responsibilities

Influence engine architecture interfaces with OSS platforms; Lead design of functional and performance enhancements to OSS platforms; Design, implement, test, and maintain engine components; Analyze technology roadmaps for OSS platforms; Partner with engineering, product, and community teams; Foster best practices in design, code reviews, and CI/CD.

Seniority

Principal, hands-on IC with mentorship