Senior Engineering Manager, AI
Core
Lead the AI Platform team to build and operate the core AI Gateway, a high-throughput system routing traffic for Large Language Models (LLMs) across the AlphaSense ecosystem.
Role type
Senior Engineering Manager, AI Infrastructure
Builds
Enterprise AI Gateway handling LLM traffic, multi-vendor orchestration, and high-scale routing/caching systems
Domain
Enterprise AI, Large Language Models (LLM), Distributed Systems
Deliverable
production ML models | infrastructure
Required skills
Engineering leadership, distributed system architecture, multi-vendor integrations, high-throughput system design, strategic communication, technical thought leadership
Preferred skills
Applied AI experience, LangChain/LangGraph/LiteLLM/OpenRouter, Temporal/Envoy, Python, Java, caching/queueing systems
Technologies
LLMs, LangChain, LangGraph, LiteLLM, OpenRouter, Temporal, Envoy, Python, Java
Responsibilities
Manage and mentor a team of up to 10 senior and staff engineers; Own strategy and architecture of the AI Gateway processing 1 Trillion LLM tokens/week; Guide integration of 6+ LLM vendors and 50+ models; Partner with product and research teams to align roadmap; Define engineering best practices and cost-tracking telemetry
Seniority
Senior, hands-on IC with team leadership