CareerPlan

Principal Data Processing Engineer-OSS

Mountain View, CaliforniaRemoteFull-time2024-09-25 → 2026-11-16
Hybrid

Core

Design, implement, and optimize high-performance data processing engine components and open-source platforms (Apache Gluten, Velox, Spark, Flink) for large-scale analytics and GenAI preprocessing.

Role type

Principal Data Processing Engineer (Individual Contributor)

Builds

Data processing engine components and integrations with open-source platforms

Domain

Distributed systems, data processing, compilers, cloud-native computing

Deliverable

production ML models

Required skills

C, C++, Java, Apache Spark, Apache Flink, Linux, performance optimization, distributed systems architecture

Preferred skills

Apache Gluten, Velox, Presto/Trino, hardware-accelerated computing

Technologies

Apache Gluten, Velox, Apache Spark, Apache Flink, C, C++, Java, Linux

Responsibilities

Influence engine architecture interfaces with OSS platforms; Lead design of functional and performance enhancements to OSS platforms; Design, implement, test, and maintain engine components; Analyze technology roadmaps to identify leadership opportunities; Partner with engineering, product, and community teams; Foster best practices in design, code reviews, and CI/CD.

Seniority

Principal, hands-on IC with mentorship