CareerPlan

Staff Software Engineer, LLM Inference Platform

Full-time2026-10-29 → 2026-11-16

Core

Technical lead for end-to-end LLM inference platform performance, optimizing token throughput and cost for models ranging from hundreds of billions to trillions of parameters.

Role type

Staff Software Engineer (LLM Inference Platform)

Builds

High-performance inference engines and disaggregated inference systems for large-scale LLMs.

Domain

Cloud Infrastructure / Large Language Model Systems

Deliverable

production ML models

Required skills

Transformer inference architecture, CUDA kernel tuning, vLLM/SGLang/TensorRT-LLM optimization, MoE parallelism, KV cache management, distributed systems design, performance benchmarking

Preferred skills

Merged PRs in vLLM, cluster-scale GPU operations

Technologies

CUDA, ROCm, HIP, vLLM, SGLang, TensorRT-LLM, NIXL, RDMA, UCX, DeepSeek, Kimi, GLM, Qwen, MiniMax

Responsibilities

Establish unified token performance metrics and load-testing benchmarks; optimize inference engine source code for scheduling and kernel execution; design Prefill-Decode disaggregation and KV transfer paths; design hybrid parallelism strategies for MoE models; write and tune critical CUDA operators and quantization evaluation pipelines.

Seniority

Staff, hands-on IC with strategic ownership