CareerPlan

Sr. Inference Engineer (LLM Serving)

PragueFull-time2026-11-19 → 2026-11-22
Hybrid

Core

Own the serving layer of GoodData Native Inference, deploying and operating open-weight LLMs and task models on GPUs to power AI answers for customers.

Role type

Senior Inference Engineer (LLM Serving)

Builds

Production inference engine, inference gateway, and model evaluation harness for analytics workloads.

Domain

AI/ML Infrastructure, LLM Serving, GPU Computing

Deliverable

production ML models

Required skills

LLM serving engines (vLLM, SGLang, TensorRT-LLM), GPU optimization (KV cache, batching, quantization), Kubernetes, Docker, Python, PyTorch, model evaluation, LoRA adapters, fine-tuning

Preferred skills

Speculative decoding, multi-LoRA serving, PEFT fine-tuning, CUDA/Triton kernels, Go/Rust, Terraform, Helm, LLM routing, embeddings, rerankers, observability (OpenTelemetry, Langfuse, Prometheus, Grafana), text-to-SQL

Technologies

vLLM, SGLang, TensorRT-LLM, Kubernetes, AWS, NVIDIA GPUs, PyTorch, gRPC, Langfuse, OpenTelemetry, Prometheus, Grafana

Responsibilities

Deploy and operate open-weight LLMs and task models on GPUs in production; tune inference parameters for latency, throughput, and cost; evaluate and select models for production; build and grow the inference gateway; adapt models using LoRA and fine-tuning for specific use cases.

Seniority

Senior, hands-on IC