Senior AI Scheduling & Orchestration Engineer
Core
Design and implement advanced batch scheduling architectures and cluster-wide admission control for high-volume AI workloads on large-scale HPC environments.
Role type
Senior IC Kubernetes scheduling engineer (AI/HPC)
Builds
Scheduling stack for multi-node gang scheduling, job queueing, and topology-aware pod placement for AI training/inference
Domain
High-performance computing (HPC) and AI cloud infrastructure
Deliverable
production ML models
Required skills
Kubernetes scheduling frameworks (Volcano, YuniKorn, Kueue), Kubernetes Dynamic Resource Allocation (DRA), custom scheduler plugins, GPU hardware architectures, distributed training frameworks (PyTorch Distributed, Ray, MPI), infrastructure automation (Terraform, Go-based Operators), topology-aware pod placement, multi-tenancy isolation
Preferred skills
Mentorship, architectural design reviews, resolving resource contention and deadlock scenarios
Technologies
Volcano, YuniKorn, Kueue, Kubernetes, NVLink, InfiniBand, MIG, Terraform, Go
Responsibilities
Design and implement batch scheduling architectures; Develop cluster-wide admission control and job queueing mechanisms; Leverage DRA and custom plugins for accelerator requests; Architect topology-aware pod placement; Implement automated GPU sharing and multi-tenancy policies; Collaborate with GPU Systems and Storage teams; Drive reliability and scalability of the scheduling stack; Mentor junior engineers and conduct design reviews
Seniority
Senior, hands-on IC