Principal Software Engineer
Core
Optimize and monitor end-to-end inference performance of OpenAI and other state-of-the-art LLMs, building software tooling to improve customer experience and reduce computing fleet footprint.
Role type
Principal Software Engineer (LLM Inference Optimization)
Builds
Software tools for LLM inference performance, porting models to new GPUs, and AI/DNN/LLM frameworks.
Domain
Artificial Intelligence / Large Language Models / High-Performance Computing
Deliverable
production ML models
Required skills
C, C++, C#, Java, JavaScript, Python, GPU architecture, HW neural net acceleration, end-to-end performance analysis, GPU profiling, DNN/LLM inference, PyTorch, Tensorflow, ONNX Runtime, CUDA, ROCm, Triton
Preferred skills
Cross-team collaboration
Technologies
Nvidia GPUs, AMD GPUs, PyTorch, Tensorflow, ONNX Runtime, CUDA, ROCm, Triton
Responsibilities
Identify and drive improvements to end-to-end inference performance of LLMs, Optimize and monitor performance of LLMs and build SW tooling, Enable fast time to market of LLMs/models and their deployments at scale, Design, implement, and test functions or components for AI/DNN/LLM frameworks, Speeding up/reducing complexity of key components/pipelines to improve performance and/or efficiency
Seniority
Principal, hands-on IC
