CareerPlan

Data Processing Engineer - I/O

Mountain View, CA / Hyderabad, IN / RemoteRemoteFull-time2025-04-02 → 2026-11-16
Hybrid

Core

Design and implement high-performance data read/write interfaces and adapters for a large-scale distributed data processing engine.

Role type

Senior IC data processing engineer (I/O subsystems)

Builds

Data processing engine components, adapters, connectors, and data format handlers

Domain

Distributed systems, large-scale data processing, storage formats

Required skills

C, C++, Rust, Linux, Apache Parquet, Apache ORC, Apache Iceberg, Apache Spark, Apache Flink, Presto/Trino, performance optimization, system architecture

Preferred skills

Rust, open table formats, catalog services

Technologies

Apache Spark, Apache Flink, Apache Parquet, Apache ORC, Apache Iceberg, Apache Arrow, Rust, C++, C, Linux

Responsibilities

Architect interfaces with data sources/sinks and catalogs; Design functional and performance enhancements for adapters/connectors; Design, implement, test, optimize, and maintain engine components; Analyze technology roadmaps for emerging data formats; Partner with product management on the data I/O roadmap; Foster best practices in design, code reviews, and CI/CD.

Seniority

Senior, hands-on IC