Principal Software Engineer - AI Inference and Serving (CoreAI)
Core
Design and optimize high-performance AI inference systems, focusing on throughput, latency, and cost efficiency for model serving.
Role type
Principal Software Engineer (AI Inference and Serving)
Builds
Secure, tenant-isolated AI inference systems with automated testing and observability
Domain
Artificial Intelligence / Distributed Systems / Model Serving
Deliverable
production ML models
Required skills
C, C++, C#, Java, JavaScript, Python, distributed services, model serving, debugging, cross-team communication, containerization, Kubernetes, vLLM, SGLang, TensorRT-LLM
Preferred skills
prefix caching, prefill/decode disaggregation, speculative decoding, quantization, tensor/expert parallelism, capacity planning, incident prevention, mentorship
Technologies
Kubernetes, vLLM, SGLang, TensorRT-LLM
Responsibilities
Evaluate and implement optimization techniques for AI inference; lead capacity planning and incident prevention; mentor engineers; build secure, tenant-isolated systems; improve services using production feedback; pursue near-linear throughput scaling.
Seniority
Principal, hands-on IC with mentorship