CareerPlan

Python Inference Engineer

RemoteFull-time2026-11-12 → 2026-11-18
Hybrid

Core

Design and deliver the inference layer of the Gcore Inference platform to support AI-driven digital experiences globally.

Role type

Senior IC Python Inference Engineer

Builds

Production inference platforms for real-time communication, streaming, and enterprise AI

Domain

Cloud infrastructure, GPU computing, and distributed machine learning systems

Deliverable

production ML models

Required skills

Python, PyTorch, Linux, Docker, Kubernetes, distributed systems, GPU computing, model optimization, cluster scheduling, debugging complex software/hardware issues

Preferred skills

vLLM, SGLang, NVIDIA Dynamo, TensorRT-LLM, CUDA, Triton, TensorRT, quantization, continuous batching, speculative decoding, prefix caching, chunked prefill, LoRA serving, profiling, distributed inference, multi-GPU systems, autoscaling, open-source contributions

Technologies

vLLM, SGLang, NVIDIA Dynamo, TensorRT-LLM, PyTorch, Kubernetes, Docker, CUDA, Triton, TensorRT

Responsibilities

Integrate and operate inference frameworks; bring new language and multimodal models into production; improve inference latency, throughput, memory use, GPU utilization, and cost efficiency; debug performance and reliability issues across model code, frameworks, GPU execution, networking, and Kubernetes; collaborate with platform, infrastructure, product, and customer-facing teams; contribute to open-source inference projects

Seniority

Senior, hands-on IC