LLM Serving Engineer (Cloud AI Engineering), Senior / Staff Engineer - Cork, Ireland
Core
Building a scalable LLM inference platform and contributing to LLM Serving packages for Qualcomm's Cloud AI team.
Role type
Senior/Staff LLM Serving Engineer (Cloud AI Engineering)
Builds
Scalable LLM inference platform and LLM Serving packages (vLLM, SGLang, TGI, Triton-Inference server, Dynamo, LLM-d)
Domain
Cloud AI, Inference Acceleration, GenAI
Deliverable
production ML models
Required skills
LLM serving/orchestration packages, foundational LLMs/VLMs/SLMs, transformer architectures, PyTorch, distributed systems, computer architecture, ML accelerators, Python, deep learning workload optimization
Preferred skills
Open-source GenAI contributions, large-scale distributed systems architecture, high-level kernel design (PyTorch, CUDA, Triton), torch.compile/torchDynamo
Technologies
vLLM, SGLang, TGI, Triton-Inference server, Dynamo, LLM-d, PyTorch, CUDA, Triton
Responsibilities
Building scalable LLM inference platform using inference techniques (disaggregated serving, KV-Cache management, parallelism, speculative algorithms, model optimization, specialized kernels); Contributing to development of LLM Serving packages; Collaborating with internal compiler, firmware, and platform teams; Understanding advanced algorithms (attention mechanisms, MoEs) to identify optimization opportunities; Driving efficient serving through autoscaling, load balancing, and routing; Engaging with open-source serving communities
Seniority
Senior/Staff, hands-on IC
