Member of Technical Staff, Inference
Core
Build and optimize the vLLM inference engine to serve LLM and diffusion models efficiently across diverse hardware.
Role type
Senior IC inference runtime engineer
Builds
High-performance inference runtime for LLMs and diffusion models
Domain
AI/ML inference systems, transformer architectures, GPU hardware
Deliverable
production ML models
Required skills
Python, PyTorch internals, transformer architectures, LLM inference systems (vLLM, TensorRT-LLM, SGLang, TGI), research paper implementation, complex codebase debugging
Preferred skills
KV-cache memory management, prefix caching, hybrid model serving, RL frameworks, multimodal inference, open-source contributions
Technologies
vLLM, PyTorch, TensorRT-LLM, SGLang, TGI
Responsibilities
Optimize model execution across diverse hardware and architectures, implement model architectures and inference techniques from research papers, contribute performant and maintainable code to the vLLM core
Seniority
Senior, hands-on IC