CareerPlan

Senior Software Engineer I, Inference

Sunnyvale / BellevueFull-time$139k–$204k2026-11-18
HybridNo visa sponsorship

Core

Lead design and architecture for a Kubernetes-native inference platform serving AI labs, startups, and enterprises.

Role type

Senior IC distributed systems engineer (AI inference)

Builds

High-throughput, low-latency inference services and orchestration systems

Domain

Cloud infrastructure + AI/ML inference

Deliverable

production ML models

Required skills

Python, Go, Kubernetes, distributed systems, performance optimization, observability (Prometheus, Grafana, OpenTelemetry), inference internals (batching, KV-cache, mixed precision)

Preferred skills

CUDA kernels, NCCL/SHARP, RDMA/NUMA, inference frameworks (vLLM, Triton, TensorRT-LLM, Ray Serve)

Responsibilities

Lead design reviews and drive architecture; define and own SLIs/SLOs; implement advanced optimizations (micro-batch schedulers, speculative decoding); strengthen incident posture (capacity planning, autoscaling); mentor IC1/IC2 engineers; own an area spanning multiple services and teams

Seniority

Senior, hands-on IC (IC3/IC4 level)