Lead AI Engineer (Gen AI & LLM Infrastructure) - New York
Core
Design, develop, and deploy scalable LLM inference, similarity search, and model governance systems to enhance customer interactions and banking processes.
Role type
Senior IC AI Engineer (Gen AI & LLM Infrastructure)
Builds
Production AI services including foundation model training, LLM inference, and vector search for Capital One's Intelligent Foundations and Experiences (IFX) team.
Domain
Financial Services / Generative AI / Large Language Models
Deliverable
production ML models
Required skills
Python, Go, Scala, Java, C++, C#, LLM Inference, Similarity Search, VectorDBs, Model Optimization, Cloud Deployment, PyTorch, AWS, Google Cloud, Azure, DevOps
Preferred skills
Experience with AWS Ultraclusters, Huggingface, Guardrails, Model Governance, Hardware utilization optimization
Technologies
AWS, Azure, Golang, Java, Python, Scala, C#, C++, PyTorch, Huggingface, VectorDBs, AWS Ultraclusters
Responsibilities
Design and deploy AI software components for foundation model training and LLM inference; Optimize training and inference software for hardware utilization, latency, and throughput; Collaborate with research scientists and product managers to deliver AI-driven products; Contribute to the strategic vision of foundational AI systems.
Seniority
Senior, hands-on IC