Senior Software Engineer, Data Infrastructure Compute
Core
Build and operate the compute layer (Kubernetes/EKS) for Roblox's data platform, managing Spark workloads, job submission, and cluster lifecycle automation.
Role type
Senior Software Engineer, Data Infrastructure Compute
Builds
Spark job submission services, interactive notebook compute, cluster lifecycle automation, and batch scheduling on Kubernetes.
Domain
Big Data, Cloud Infrastructure, Distributed Systems
Deliverable
production ML models
Required skills
Apache Spark on Kubernetes, Kubernetes operations, Go/Java/Scala/Clojure, AWS/GCP, Infrastructure as Code, SRE practices, Technical leadership
Preferred skills
YuniKorn/Volcano/Kueue, Spark Connect/Livy/Kyuubi, Spark Operator/Karpenter/Helm/Terraform/Argo, Native execution engines (Comet/Gluten/Velox), JVM tuning, Hadoop to Kubernetes migration
Responsibilities
Design and operate Spark job submission APIs and libraries; Launch Spark Connect for notebook workflows; Build fleet management automation for EKS; Develop auto-tuners for Spark performance; Operate YuniKorn-based batch scheduling; Evaluate emerging technologies like Apache Comet; Define SLOs and drive cost optimization; Partner with data engineering and ML teams.
Seniority
Senior, hands-on IC with technical leadership