Principal Software Engineer, Inference
Core
Lead the model runtime architecture for HPE AI Essentials, an inference platform enabling enterprises to operate large language models on customer-owned hardware in edge-to-cloud and sovereign environments.
Role type
Principal Software Engineer (LLM Inference Runtime)
Builds
High-performance LLM inference runtime engine and Kubernetes orchestration layer
Domain
Enterprise AI / Large Language Model Inference / Cloud Infrastructure
Deliverable
production ML models
Required skills
LLM inference engines (vLLM, SGLang, TensorRT-LLM), continuous batching, KV cache management, tensor/pipeline parallelism, Kubernetes operators, Go, Python, C++/CUDA profiling
Preferred skills
Upstream contributions to inference runtimes, disaggregated prefill/decode, RDMA/GPUDirect Storage, MIG/fractional GPU allocation
Technologies
vLLM, SGLang, TensorRT-LLM, Kubernetes, NCCL, CUDA, RDMA, InfiniBand
Responsibilities
Define technical direction for LLM serving deployment including engine integration and distributed execution; Partner with performance teams to optimize latency and throughput; Evaluate and adopt emerging runtimes and quantization schemes; Define orchestration layer for model admission and GPU scheduling; Mentor engineers and lead architecture reviews.
Seniority
Principal, hands-on IC with strategic leadership
