Senior IA Inference Engineer (x/f/m)
Core
Build and operate a self-hosted LLM serving platform for healthcare AI products, managing GPU fleets and model deployment to ensure data residency and performance.
Role type
Senior Inference Engineer (ML Platform)
Builds
Shared LLM serving foundations, model registry, and inference stack for AI features
Domain
Healthcare + Large Language Model (LLM) Infrastructure
Deliverable
production ML models
Required skills
LLM serving engines (vLLM, SGLang, TensorRT-LLM, Triton), Python, Terraform, Kubernetes, GPU workload optimization, MLOps, observability, incident response
Preferred skills
Distributed serving frameworks (KServe, llm-d), GPU kernel experience (CUDA, Triton), regulated environment compliance (GDPR, HDS)
Technologies
vLLM, Triton Inference Server, TensorRT-LLM, Ray Serve, Anyscale, Kubernetes, GCP/AWS/Azure, Python, Terraform, Docker, Helm, Datadog, MLflow, NVIDIA GPUs (L4, A100, H200)
Responsibilities
Define serving stack and GPU capacity planning, manage model registry and promotion paths, optimize inference performance and latency, ensure production reliability and disaster recovery, build cost and performance dashboards, maintain security and compliance of the inference stack
Seniority
Senior, hands-on IC
