Senior Performance Engineer - LLM Inference Frameworks
Core
Design, implement, and optimize high-performance inference pipelines for large language models running on GPUs.
Role type
Senior IC performance engineer (LLM inference frameworks)
Builds
High-throughput, memory-efficient LLM inference frameworks and pipelines
Domain
Generative AI, GPU computing, LLM inference optimization
Deliverable
production ML models
Required skills
Python, software design, deep learning frameworks (PyTorch, HuggingFace), performance profiling and debugging, GPU utilization optimization, memory management, speculative decoding, context caching, quantization (FP8/INT4)
Preferred skills
Contributions to inference frameworks (TensorRT-LLM, vLLM, SGLang), performance modeling, distributed model execution, NVIDIA profiling tools (Nsight Systems, PyTorch Profiler)
Technologies
PyTorch, HuggingFace, TensorRT-LLM, Nsight Systems, FP8, INT4
Responsibilities
Design and implement high-performance inference pipelines; Profile and tune model execution across the stack; Design memory management strategies; Innovate techniques like Speculative Decoding and quantization; Develop benchmarking and testing systems
Seniority
Senior, hands-on IC
