Research Engineer, Inference Foundation
Core
Build and optimize the core inference stack (engine and orchestrator) for Mistral's production LLMs, focusing on throughput, latency, and capacity elasticity.
Role type
Senior IC systems engineer (LLM inference)
Builds
High-performance inference engine, orchestration platform, and serving infrastructure for frontier models
Domain
AI/ML infrastructure, distributed systems, GPU computing
Deliverable
production ML models
Required skills
Inference engine internals (prefill/decode, KV-cache, batching, speculative decoding), distributed serving architectures, CUDA/NCCL, Kubernetes, Python, PyTorch
Preferred skills
vLLM/sglang contributions, MoE model serving, CUDA/Triton kernel development, Rust/C++
Technologies
vLLM, SGLang, TensorRT-LLM, Kubernetes, CUDA, NCCL, InfiniBand/RDMA, PyTorch, Python
Responsibilities
Develop and fix the inference engine and orchestrator, optimize serving efficiency across the fleet, build serving infrastructure for RL and post-training loops
Seniority
Senior, hands-on IC