LLM Infrastructure Engineer
Core
Design, deploy, and scale infrastructure for training, fine-tuning, and serving large language models.
Role type
Senior IC LLM Infrastructure Engineer
Builds
Production ML infrastructure for LLM training and inference
Domain
Cloud systems, GPU computing, distributed systems
Deliverable
production ML models
Required skills
Python, Kubernetes, Docker, distributed systems, GPU computing, performance optimization, infrastructure-as-code, CI/CD
Preferred skills
AWS (EC2, EKS, S3, SageMaker), LLM serving frameworks (vLLM, TensorRT-LLM, Triton), Terraform, CloudFormation, distributed training frameworks (PyTorch Distributed, DeepSpeed, Ray)
Technologies
AWS, Python, Docker, Kubernetes, Terraform, CloudFormation, PyTorch, vLLM, TensorRT-LLM, Triton Inference Server
Responsibilities
Design and maintain infrastructure for LLM training and serving; Deploy and scale inference services; Manage GPU compute resources; Build data pipelines for model training; Implement monitoring and alerting; Optimize infrastructure costs; Automate deployment workflows; Collaborate with ML teams to move models to production; Resolve infrastructure issues
Seniority
Senior, hands-on IC