Sr. Inference Engineer (LLM Serving)
Core
Own the serving layer of GoodData Native Inference, deploying and operating open-weight LLMs and task models on GPUs to power AI answers for customers.
Role type
Senior Inference Engineer (LLM Serving)
Builds
Production inference engine, inference gateway, and model evaluation harness for analytics workloads.
Domain
AI/ML Infrastructure, LLM Serving, GPU Computing
Deliverable
production ML models
Required skills
LLM serving engines (vLLM, SGLang, TensorRT-LLM), GPU optimization (KV cache, batching, quantization), Kubernetes, Docker, Python, PyTorch, model evaluation, LoRA adapters, fine-tuning
Preferred skills
Speculative decoding, multi-LoRA serving, PEFT fine-tuning, CUDA/Triton kernels, Go/Rust, Terraform, Helm, LLM routing, embeddings, rerankers, observability (OpenTelemetry, Langfuse, Prometheus, Grafana), text-to-SQL
Technologies
vLLM, SGLang, TensorRT-LLM, Kubernetes, AWS, NVIDIA GPUs, PyTorch, gRPC, Langfuse, OpenTelemetry, Prometheus, Grafana
Responsibilities
Deploy and operate open-weight LLMs and task models on GPUs in production; tune inference parameters for latency, throughput, and cost; evaluate and select models for production; build and grow the inference gateway; adapt models using LoRA and fine-tuning for specific use cases.
Seniority
Senior, hands-on IC