Senior Software Engineer I, Inference
Core
Lead design and architecture for a Kubernetes-native inference platform serving AI labs, startups, and enterprises.
Role type
Senior IC distributed systems engineer (AI inference)
Builds
High-throughput, low-latency inference services and orchestration systems
Domain
Cloud infrastructure + AI/ML inference
Deliverable
production ML models
Required skills
Python, Go, Kubernetes, distributed systems, performance optimization, observability (Prometheus, Grafana, OpenTelemetry), inference internals (batching, KV-cache, mixed precision)
Preferred skills
CUDA kernels, NCCL/SHARP, RDMA/NUMA, inference frameworks (vLLM, Triton, TensorRT-LLM, Ray Serve)
Responsibilities
Lead design reviews and drive architecture; define and own SLIs/SLOs; implement advanced optimizations (micro-batch schedulers, speculative decoding); strengthen incident posture (capacity planning, autoscaling); mentor IC1/IC2 engineers; own an area spanning multiple services and teams
Seniority
Senior, hands-on IC (IC3/IC4 level)