LLM Inference Platform Architect
Core
Design and lead a secure, internal LLM serving platform on fully controlled GPU hardware to ensure data privacy and independence from external providers.
Role type
Senior IC infrastructure architect (LLM inference platform)
Builds
Internal LLM serving platform, GPU clusters, and inference engines
Domain
Quantum AI, enterprise infrastructure, regulated industries
Deliverable
production ML models
Required skills
Kubernetes, LLM inference engines (vLLM, SGLang, TensorRT-LLM), Infrastructure as Code (Terraform), CI/CD, observability, security design, network design, architecture guidelines
Preferred skills
Enterprise-grade GPU management (NVIDIA/AMD), CUDA/ROCm, banking/regulated industry experience, DORA compliance
Technologies
Kubernetes, vLLM, SGLang, TensorRT-LLM, Terraform, NVIDIA GPUs, AMD Instinct, CUDA, ROCm
Responsibilities
Design and manage the full serving stack including GPU clusters and inference engines; set platform standards and ensure regulatory compliance; partner with security and operations to deploy optimized models
Seniority
Senior, hands-on IC