Hadoop Software Engineer
Core
Build and optimize eBay's next-generation lakehouse platform, focusing on enhancing Apache Spark stability/performance and extending Apache Iceberg capabilities.
Role type
Senior IC Hadoop/Spark Software Engineer
Builds
Internal storage and metadata services for lakehouse architecture; production-grade software using AI coding agents
Domain
Ecommerce, Big Data, Distributed Systems
Deliverable
production ML models | product features
Required skills
Java or Scala, Apache Spark internals (Catalyst, Tungsten, shuffle, AQE, memory management), Apache Iceberg/Delta/Hudi internals, Linux/Unix shell scripting, distributed system design, AI coding agent prompt engineering
Preferred skills
Open source contributions to Apache Spark/Iceberg, HDFS/HBase/Yarn experience
Technologies
Apache Spark, Apache Iceberg, Apache Parquet, Apache Hudi, Delta Lake, HDFS, HBase, Yarn, Linux, AI coding agents (Claude Code, Codex)
Responsibilities
Troubleshoot and resolve production issues to ensure minimal downtime; maintain and improve system monitoring, logging, and alerting; build and optimize lakehouse platform components
Seniority
Senior, hands-on IC
