Principal Data Processing Engineer-OSS
Core
Principal Data Processing Engineer leading the design, implementation, and optimization of open-source data processing platforms (Apache Gluten, Velox, Spark, Flink) for a high-performance data engine.
Role type
Principal IC Data Processing Engineer (Open Source)
Builds
High-performance data processing engine components and open-source platform integrations
Domain
Distributed systems, data processing, compilers, C++/Java systems programming
Deliverable
production ML models
Required skills
C++, Java, Apache Spark, Apache Flink, Linux, performance optimization, distributed systems architecture
Preferred skills
Apache Gluten, Velox, Presto/Trino, hardware-accelerated computing
Technologies
Apache Gluten, Velox, Apache Spark, Apache Flink, C++, Java, Linux
Responsibilities
Influence engine architecture interfaces with OSS platforms; Lead design of functional and performance enhancements to OSS platforms; Design, implement, test, and maintain engine components; Analyze technology roadmaps for OSS platforms; Partner with engineering, product, and community teams; Foster best practices in design, code reviews, and CI/CD.
Seniority
Principal, hands-on IC with mentorship