Software Engineer, AI Infrastructure
Core
Design and build scalable backend infrastructure for distributed training, inference, and data pipelines to power Fireworks AI's generative AI platform.
Role type
Senior IC software engineer (AI infrastructure)
Builds
LLM CI/CD pipelines, control planes, model serving systems, and performance optimization frameworks
Domain
Generative AI / Large Language Model (LLM) inference infrastructure
Deliverable
production ML models
Required skills
Python, Go, ML infrastructure, Kubernetes, distributed systems, LLM knowledge (context length, KV cache), CI/CD
Preferred skills
vLLM, Sglang, TRT-LLM, open-source contributions, large-scale MLOps
Technologies
PyTorch, MLflow, Vertex AI, SageMaker, Kubernetes, vLLM, Sglang, TRT-LLM
Responsibilities
Design scalable backend infrastructure for distributed training and inference; Build and maintain core backend services; Support performance optimization and cost efficiency; Build frameworks for model quality; Collaborate with product and performance teams; Participate in code reviews and CI/CD processes
Seniority
Mid-level, hands-on IC
