Software Engineer - Model Performance Systems
Core
Building automated benchmarking, profiling, and optimization tooling for LLM inference performance and developer environments.
Role type
Senior IC software engineer (LLM performance systems)
Builds
Automated performance suites, GPU-enabled dev environments, open-source benchmarking tools, and real-time observability dashboards
Domain
AI Infrastructure / High-Performance Computing / LLM Inference
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Python, C++, GPU memory subsystems, InfiniBand, PyTorch Profiler, NVIDIA Nsight Systems, py-spy, CI/CD automation, system profiling, benchmarking automation
Preferred skills
NVIDIA software stack mastery, stress-testing, fuzzy testing, Transformer math understanding
Technologies
PyTorch, NVIDIA Nsight Systems, py-spy, InfiniBand, GitHub Codespaces
Responsibilities
Evaluate and automate LLM quality benchmarks (GSM8K, MMLU) and custom performance suites; Develop internal GPU-enabled development environments; Build open-source tools for model evaluation and analysis; Profile compute/networking stacks to identify bottlenecks; Develop real-time dashboards for system health and runtime performance; Automate performance testing via CI/CD pipelines; Build tools to identify optimal latency/cost/quality configurations
Seniority
Mid-senior, high leverage, lead of a small team