CareerPlan

GenAI Platform Engineer – LLM Gateway

IndiaRemoteFull-time2026-11-18 → 2026-11-19
Remote · from India

Core

Build, operate, and improve an enterprise-scale GenAI platform and LLM Gateway providing secure, scalable access to multiple large language model providers.

Role type

Senior individual contributor GenAI Platform Engineer (LLM Gateway)

Builds

Enterprise LLM Gateway, multi-provider model integrations, agentic AI solutions, and open-source foundations

Domain

Generative AI / Large Language Models / Cloud Infrastructure

Deliverable

production ML models

Required skills

Python backend development, asynchronous programming, LLM provider API integration, tool calling, AWS containerization, CI/CD pipelines, observability, security engineering, root-cause analysis

Preferred skills

AI/LLM gateway experience, open-source software operations, agentic AI architectures, LangGraph/LangChain, Kubernetes, Infrastructure as Code, frontend development, AI-assisted development tools

Technologies

Python, FastAPI, OpenAI, Azure OpenAI, AWS Bedrock, Docker, GitHub Actions, OpenTelemetry, Langfuse, MCP, LangGraph, LangChain, Kubernetes, Terraform, React, TypeScript

Responsibilities

Design and develop enterprise LLM Gateway features; Build and maintain multi-provider model integrations and intelligent routing; Develop platform features for usage metering and cost transparency; Integrate agentic AI capabilities; Develop and maintain Model Context Protocol (MCP) servers; Maintain and enhance the platform's open-source foundations; Build automated quality gates within CI/CD pipelines; Implement observability solutions; Monitor platform performance and resolve infrastructure issues; Participate in production support and on-call rotations; Conduct blameless postmortems; Apply secure-by-default engineering practices; Maintain dependency security; Evaluate emerging generative AI technologies; Support internal engineering teams in onboarding; Contribute to InnerSource initiatives; Identify opportunities to improve platform capabilities

Seniority

Senior, hands-on IC