Senior Systems Software Engineer - Accelerated Spark Applications
Core
Design, build, and operate a hosted Apache Spark service control plane on Kubernetes to accelerate data processing on GPUs for NVIDIA's internal data platform.
Role type
Senior Systems Software Engineer (Distributed Systems & GPU Acceleration)
Builds
API-based Spark service, cuDF Spark plugin, CPU/GPU workload router, and service control plane on Kubernetes.
Domain
Big Data, Distributed Systems, GPU Acceleration, Cloud Infrastructure
Required skills
Java, Scala, C++, Python, Kubernetes, REST APIs, Distributed Systems, On-call support, AI tools
Preferred skills
Apache Spark internals, Spark Connect, Kyuubi, Delta Lake, Apache Iceberg, Unity Catalog, CUDA, cuDF
Technologies
Kubernetes, Helm, Flux, Spark Connect, Kyuubi, cuDF, AWS, Delta Lake, Apache Iceberg, Unity Catalog
Responsibilities
Design and operate the Spark service control plane; own code reviews for services; extend the cuDF Spark runtime and workload router; investigate and fix production failures; maintain metrics, dashboards, and on-call runbooks; provide infrastructure and CI/CD recommendations.
Seniority
Senior, hands-on IC