AI Engineer (LLM Gateway, FM Hosting) - New York
Core
Design, develop, test, deploy, and support AI software components including foundation model training, LLM inference, agents, and multi-agent workflows to deliver AI-powered products for associates and customers.
Role type
Senior IC AI Engineer (LLM Gateway, FM Hosting)
Builds
Scalable, responsible AI systems and proprietary solutions serving millions of customers
Domain
Banking + Large Language Models (LLM) & Foundation Models
Deliverable
production ML models
Required skills
Python, Go, Scala, CUDA, Java, LLM inference, similarity search, vector databases, guardrails, model evaluation, distributed systems, GPU/TPU optimization
Preferred skills
Leading AI system development, deploying scalable AI on cloud platforms, designing agentic AI systems, optimizing training/inference software, building AI/ML technologies, defining AI model governance, influencing architecture across product lines
Technologies
AWS Ultraclusters, Hugging Face, PyTorch, AWS, Azure, C#, Golang, C++, C#
Responsibilities
Partner with engineers and product managers to deliver AI-powered products; Design and optimize foundation model training and LLM inference pipelines; Define and maintain AI reliability SLOs; Lead cross-functional technical reviews for security and compliance; Mentor Principal and Senior Associates in scalable design and performance tuning
Seniority
Senior, hands-on IC with mentorship responsibilities
