CareerPlan

Software Engineer II & Senior Software Engineer - AI Inference and Serving (CoreAI)

United States, Washington, RedmondFull-time2026-11-17 → 2026-11-19

Core

Optimize AI inference systems for profile execution, memory management, and dependency bottlenecks to improve throughput, latency, and cost.

Role type

Senior IC software engineer (AI inference and serving)

Builds

Secure, tenant-isolated model-serving runtimes and gateways

Domain

AI/ML infrastructure, high-performance computing

Deliverable

production ML models

Required skills

C++, C#, Rust, Go, Java, Python, networking, concurrency, debugging, containerization, Kubernetes, vLLM, SGLang, TensorRT-LLM

Preferred skills

cache-aware routing, admission control, continuous batching, KV-cache management, load testing, failure testing, observability, automated testing, reproducible benchmarking, near-linear throughput scaling

Responsibilities

Improve cache-aware routing, admission control, continuous batching, or KV-cache management; Validate time-to-first-token, tail latency, and cost per token; Build secure, tenant-isolated systems with automated testing and observability; Participate in operational support and improve services using production feedback; Preserve model quality and numerical correctness; Use reproducible benchmarks to improve throughput, latency, and cost; Pursue near-linear throughput scaling across GPU and node configurations

Seniority

Senior, hands-on IC