Python Inference Engineer
Core
Design and deliver the inference layer of the Gcore Inference platform to support AI-driven digital experiences globally.
Role type
Senior IC Python Inference Engineer
Builds
Production inference platforms for real-time communication, streaming, and enterprise AI
Domain
Cloud infrastructure, GPU computing, and distributed machine learning systems
Deliverable
production ML models
Required skills
Python, PyTorch, Linux, Docker, Kubernetes, distributed systems, GPU computing, model optimization, cluster scheduling, debugging complex software/hardware issues
Preferred skills
vLLM, SGLang, NVIDIA Dynamo, TensorRT-LLM, CUDA, Triton, TensorRT, quantization, continuous batching, speculative decoding, prefix caching, chunked prefill, LoRA serving, profiling, distributed inference, multi-GPU systems, autoscaling, open-source contributions
Technologies
vLLM, SGLang, NVIDIA Dynamo, TensorRT-LLM, PyTorch, Kubernetes, Docker, CUDA, Triton, TensorRT
Responsibilities
Integrate and operate inference frameworks; bring new language and multimodal models into production; improve inference latency, throughput, memory use, GPU utilization, and cost efficiency; debug performance and reliability issues across model code, frameworks, GPU execution, networking, and Kubernetes; collaborate with platform, infrastructure, product, and customer-facing teams; contribute to open-source inference projects
Seniority
Senior, hands-on IC