Staff Software Engineer - Lakehouse
Core
Building and optimizing a next-generation lakehouse platform by enhancing Apache Spark stability, usability, and performance, and extending Apache Iceberg capabilities.
Role type
Staff Software Engineer (Distributed Systems & Data Platform)
Builds
Internal storage and metadata services, production-grade Spark/Iceberg platform features
Domain
Ecommerce, Big Data, Distributed Systems
Deliverable
production ML models | infrastructure
Required skills
Java or Scala, Apache Spark internals (Catalyst, Tungsten, shuffle, AQE, memory management), Apache Iceberg/Delta/Hudi internals, Linux/Unix shell scripting, distributed system design, networking principles
Preferred skills
Open source contributions (Apache Spark/Iceberg), HDFS/HBase/Yarn experience
Technologies
Apache Spark, Apache Iceberg, Apache Parquet, Java, Scala, Linux, Hadoop
Responsibilities
Enhance Spark stability, usability, and performance; extend Iceberg capabilities; build internal storage and metadata services; troubleshoot and resolve production issues; maintain system monitoring, logging, and alerting
Seniority
Staff, hands-on IC with architectural impact
