AI Infrastructure, Inference
Core
Build and maintain tools and systems enabling Microsoft AI researchers to run models efficiently for RL, synthetic data generation, and evals.
Role type
Senior IC AI Infrastructure Engineer (Inference)
Builds
Compute-efficient inference systems, distributed tools, and profiling/debugging utilities for generative AI models.
Domain
Generative AI, Distributed Systems, High-Performance Computing
Deliverable
production ML models
Required skills
Generative AI architecture optimization, Open-source inference frameworks (vLLM, SGLang), Python ecosystem, GPU kernel programming, Distributed computing, PyTorch benchmarking and profiling, C/C++
Preferred skills
JAX scaling, Large-scale production inference, Pybind/nanobind, FastAPI
Technologies
vLLM, SGLang, PyTorch, JAX, Python, C++, CUDA, Infiniband, NVLink
Responsibilities
Implement frontier AI research ideas, Introduce systems to improve inference performance, Build tools to debug bottlenecks and distributed issues, Establish processes to enhance team productivity, Overcome roadblocks to deliver work iteratively
Seniority
Senior, hands-on IC