腾讯云-MaaS平台推理流量调度高级工程师(北/杭/深)
Core
Design and implement traffic scheduling and distribution strategies for MaaS platform inference across regions and clusters to optimize GPU utilization and load balancing.
Role type
Senior IC distributed systems engineer (LLM inference traffic scheduling)
Builds
MaaS platform inference traffic scheduling system, tenant-level rate limiting and quota management, elastic orchestration, and inference performance optimization tools.
Domain
Cloud computing + Large Language Model (LLM) inference
Deliverable
production ML models
Required skills
Distributed systems, high concurrency, load balancing, service governance, LLM inference engine architecture, Prefill/Decode, continuous batching, KV Cache, prefix caching, Golang, TCP/IP, HTTP, streaming protocols
Preferred skills
Large-scale distributed system experience, LLM inference optimization, inference engine tuning or secondary development
Responsibilities
Design scheduling strategies for cross-region and cross-cluster inference traffic; Build tenant-level rate limiting and quota management systems; Implement elastic orchestration capabilities including HPA and dynamic scaling; Optimize inference link performance via caching-aware routing and batch processing.
Seniority
Senior, hands-on IC
