Software Engineer, LLM Infrastructure
Core
Design and develop scalable backend infrastructure for distributed training, inference, and data pipelines to power Fireworks AI's generative AI platform.
Role type
Senior IC software engineer (LLM infrastructure)
Builds
LLM CI/CD pipelines, control planes, model serving systems, and performance optimization frameworks
Domain
AI infrastructure / Machine Learning Systems
Deliverable
production ML models
Required skills
Python, Go, ML infrastructure, Kubernetes, distributed systems, performance optimization, CI/CD
Preferred skills
vLLM, Sglang, TRT-LLM, open-source contributions, large-scale MLOps
Technologies
PyTorch, MLflow, Vertex AI, SageMaker, Kubernetes, vLLM, Sglang, TRT-LLM
Responsibilities
Design scalable backend infrastructure for distributed training and inference; Build and maintain core backend services; Optimize performance, cost, and reliability across compute/storage/networking; Develop frameworks for model quality; Collaborate on translating research needs into infrastructure solutions; Participate in code reviews and CI/CD processes
Seniority
Senior, hands-on IC