CareerPlan

Principal, AI Platform Engineering

New York, NYFull-time2026-07-09 → 2026-11-16

Core

Design and build an enterprise-grade generative AI platform enabling teams to build, deploy, and operationalize AI applications with security and compliance.

Role type

Principal AI Platform Engineer (Generative AI Infrastructure)

Builds

Multi-LLM gateway, RAG retrieval services, model registry, prompt library, MCP gateway, and FinOps infrastructure.

Domain

Financial Services / Generative AI Platform Engineering

Deliverable

production ML models

Required skills

Generative AI & LLM architecture, RAG systems, Python/Go/Rust/Java, Model serving (vLLM, Ollama), Vector databases, Cloud platforms (AWS/GCP/Azure), Kubernetes, Security architecture (RBAC, ABAC, Encryption), Compliance frameworks (SOC 2, ISO 27001, GDPR), CI/CD for AI, Observability.

Preferred skills

Financial services tech experience, LangChain/LlamaIndex, MLOps tools, Eval frameworks, Zero-trust architecture, Open-source AI contributions.

Technologies

vLLM, Ollama, TensorFlow Serving, Docker, Kubernetes, AWS, GCP, Azure, LangChain, LlamaIndex.

Responsibilities

Architect multi-LLM gateway and RAG services; Design model registry and governance; Implement secure API gateways and secrets management; Lead platform strategy and roadmap; Build CI/CD pipelines and observability; Ensure compliance with regulatory requirements.

Seniority

Principal, hands-on IC with strategic leadership