Platform Engineer - Self Hosting LLMs built with GO
Core
Build the LLM access layer for agents on the PhysicsX platform, serving self-hosted open models and routing tasks to the best-fitting models.
Role type
Senior Platform Engineer (LLM Serving & Routing)
Builds
Self-hosted LLM serving infrastructure and flexible routing foundation for the PhysicsX platform
Domain
AI/ML Infrastructure, Industrial Simulation Software
Deliverable
production ML models
Required skills
vLLM, SGLang, KServe, Go, Python, Kubernetes, GPU workloads, model routing, inference optimization, autoscaling, observability
Preferred skills
Mixture-of-Experts, speculative decoding, confidential computing, Kata Containers, CUDA profiling
Technologies
vLLM, SGLang, KServe, Kubernetes, Go, Python
Responsibilities
Operate inference engines under production constraints, own LLM packaging and deployment lifecycle, tune batching and parallelism, build autoscaling and capacity management, make routing decisions observable and explainable, integrate with identity and observability systems
Seniority
Senior, hands-on IC
