Software Engineer - AI Performance
Core
Design and build infrastructure to benchmark, profile, and tune AI agents and model-serving systems in enterprise SaaS environments.
Role type
Senior Software Engineer (AI Performance & Benchmarking)
Builds
Benchmarking infrastructure, agent evaluation pipelines, and telemetry systems for AI model performance.
Domain
Enterprise SaaS, AI/LLM Systems, Cloud Infrastructure
Deliverable
production ML models | infrastructure
Required skills
Backend engineering, API design, Cloud-native architecture, Python, Observability/Profiling, Data pipeline development
Preferred skills
Vector search, Model tuning (LoRA, DPO), Open-source contributions, Kubernetes
Technologies
vLLM, Triton, Ray, Kubernetes, AWS, GCP, Azure, Grafana, Prometheus, ELK
Responsibilities
Build and scale benchmarking infrastructure for evaluating AI systems; Design agent evaluation pipelines measuring reasoning and accuracy; Collaborate with AI/Infra teams to instrument model-serving systems; Develop data pipelines capturing model performance and latency; Contribute to open benchmarking frameworks; Ensure benchmarks reflect real-world customer value.
Seniority
Senior, hands-on IC
