Principal Data Processing Engineer
Core
Lead the architecture, design, and implementation of core execution components for a high-performance, hardware-accelerated data processing engine.
Role type
Principal Data Processing Engineer (IC + Technical Lead)
Builds
Enterprise-grade data processing engine components for large-scale analytics and GenAI preprocessing
Domain
Distributed systems, parallel processing, compilers, and cloud-native computing
Deliverable
production ML models
Required skills
C, C++, Rust, Linux development, parallel/distributed execution engine architecture, query optimization, hardware-accelerated computing, system design, technical leadership
Preferred skills
Apache Spark, Apache Flink, Apache Doris, Apache Gluten, Velox, Apache DataFusion/Comet
Technologies
Apache Spark, Apache Flink, Apache Doris, Apache Gluten, Velox, Apache DataFusion, Comet, C, C++, Rust, Linux
Responsibilities
Drive evolution of parallel and distributed execution engine architecture; lead end-to-end lifecycle of design, implementation, and rollout of engine components; design, implement, test, and maintain critical engine components; analyze technology advances to enhance product leadership; mentor engineers on the team; foster best practices in design, code reviews, testing, and CI/CD
Seniority
Principal, hands-on IC with technical leadership