CareerPlan

Senior Software Engineer — Distributed Compute / Spark Systems

Bay Area OfficeFull-time2026-11-11 → 2026-11-22
On-site

Core

Build distributed compute systems for enterprise-scale data and AI workloads, optimizing performance, scheduling, and cost at petabyte/exabyte scale.

Role type

Senior IC distributed systems engineer (query/runtime systems)

Builds

Core infrastructure for Crunch, Granica's continuous optimization product for enterprise lakehouse data

Domain

Data infrastructure + AI systems

Deliverable

production ML models

Required skills

distributed execution, query planning, scheduling, resource management, fault tolerance, performance optimization, Spark internals (Catalyst, AQE, shuffle, joins, aggregation, spill, memory management), systems programming (Scala, Java, Go, Rust, C++)

Preferred skills

cost-based optimization, vectorized execution, workload schedulers, execution control planes, storage-aware execution, adaptive query optimization

Technologies

Apache Spark, Trino, Presto, Flink, Iceberg, Delta Lake, Hudi, cloud object stores

Responsibilities

Build and optimize distributed compute infrastructure for large-scale analytical and AI workloads; Improve execution performance across scans, joins, aggregations, shuffles, spills, caching, partitioning, and task scheduling; Design adaptive systems for workload routing, execution planning, resource allocation, and cluster efficiency; Debug bottlenecks across CPU, memory, network, storage, metadata, and distributed execution layers; Improve reliability, fault tolerance, and compute efficiency across Spark and adjacent systems

Seniority

Senior, hands-on IC