Staff Software Engineer, Data Infrastructure
Core
Designing and operating the backbone of Instacart's data ecosystem, including open lakehouse foundations, multi-engine compute platforms, and real-time streaming infrastructure.
Role type
Staff Software Engineer, Data Infrastructure
Builds
Open data lakehouse on Apache Iceberg, multi-engine compute platform for stream and analytical workloads, self-serve tooling for Data Science/ML/Ads/Finance teams
Domain
Grocery delivery / Data Infrastructure / Distributed Systems
Deliverable
production ML models | infrastructure
Required skills
Distributed systems architecture, Open table formats (Apache Iceberg, Delta Lake), Distributed query/compute engines (Trino, Spark, ClickHouse), Event-driven/streaming infrastructure (Kafka, Flink), Platform migration strategy, Cost/benefit modeling, Technical leadership
Preferred skills
Data governance controls, Compliance frameworks (SOX, CPRA, GDPR), FinOps, Orchestration (Airflow), Data transformation (dbt)
Technologies
Apache Iceberg, Apache Flink, Trino, ClickHouse, Apache Kafka, Apache Spark, Snowflake, Databricks, Confluent, Airflow, dbt, Delta Lake, Scala, Python, Postgres, AWS
Responsibilities
Translate data strategy into multi-year architecture roadmap; Own open lakehouse foundation and governance; Drive real-time and streaming infrastructure for critical use cases; Pioneer AI-native data infrastructure engineering; Lead architecture reviews and mentor senior/staff engineers
Seniority
Staff, hands-on IC with strategic scope
