Senior Software Engineer I, Inference
Core
Senior Software Engineer leading design, architecture, and optimization of a Kubernetes-native inference platform for AI workloads.
Role type
Senior IC distributed systems engineer (AI inference)
Builds
Kubernetes-native inference platform serving AI labs, startups, and enterprises
Domain
Cloud infrastructure / AI inference
Deliverable
production ML models
Required skills
distributed systems, Kubernetes, Python, Go, networked systems, performance optimization, observability, inference internals (batching, caching, mixed precision)
Preferred skills
CUDA kernels, NCCL/SHARP, RDMA/NUMA, GPU interconnect topologies, inference frameworks (vLLM, Triton, TensorRT-LLM, Ray Serve, TorchServe)
Technologies
Kubernetes, Prometheus, Grafana, OpenTelemetry, Python, Go, C++, CUDA, NCCL, RDMA
Responsibilities
Lead design reviews and drive architecture; define and own SLIs/SLOs; implement advanced optimizations (micro-batch schedulers, speculative decoding, KV-cache reuse); strengthen incident posture (capacity planning, autoscaling, graceful degradation); mentor IC1/IC2 engineers; own an area spanning multiple services and teams
Seniority
Senior, hands-on IC (IC3/IC4 level)
