Senior Staff Inference Performance Engineer
Core
Own end-to-end performance of AI inference workloads on an orchestrated cloud platform, ensuring performance matches bare metal standards across orchestration and inference internals.
Role type
Senior Staff Inference Performance Engineer
Builds
Deployment and orchestration technology for d-Matrix's internal and customer-facing inference services
Domain
Generative AI, LLM inference serving, high-performance computing
Deliverable
production ML models
Required skills
Linux internals, NUMA, CPU pinning, PCIe, memory/cache behavior, network paths, Kubernetes performance tuning, LLM inference serving (batching, KV-cache, prefill/decode), Python, C/C++
Preferred skills
LLM serving frameworks (vLLM, SGLang, TensorRT-LLM), speculative decoding, heterogeneous serving architectures, GPU programming (CUDA, Nsight, NCCL), Rust
Technologies
Kubernetes, d-Matrix Corsair accelerators, GPUs
Responsibilities
Diagnose performance problems across the full serving path, isolate issues to specific layers/owners, write reproducible problem statements, protect throughput/latency targets, assess fleet scaling requirements, capture node requirements in orchestration config, build telemetry/benchmarks, create playbooks and mentor engineers
Seniority
Senior Staff, hands-on IC
