推理框架研发工程师-国产GPU方向-深圳/北京/上海
Core
Develop and optimize large model inference engines and PD-separated inference scheduling systems to enhance the efficiency of large-scale distributed inference systems.
Role type
Senior IC machine-learning engineer (inference optimization)
Builds
High-performance distributed inference systems for large language models
Domain
AI/ML, Large Language Models, GPU Computing
Deliverable
production ML models
Required skills
C/C++, Python, large model inference frameworks (vLLM, SGLang, TensorRT-LLM), parallel strategies (data parallelism, pipeline parallelism), deep learning operator implementation, GPU/AI chip architecture knowledge
Preferred skills
NVLINK and GPU RDMA communication experience, system performance analysis and tuning
Technologies
vLLM, SGLang, TensorRT-LLM, NVLINK, GPU RDMA
Responsibilities
Develop and optimize large model inference engines and PD-separated inference scheduling systems; Support mainstream GPUs and heterogeneous AI chips to optimize large model inference performance.
