Principal consultant GenAI LLM Ops Engineer
Core
Build, operate, and scale Large Language Model (LLM) platforms and applications, owning the end-to-end operational lifecycle from deployment to optimization.
Role type
Principal consultant GenAI LLM Ops Engineer
Builds
Robust, compliant, and efficient LLM-powered experiences for global enterprises
Domain
Generative AI, Large Language Models, Cloud Infrastructure
Deliverable
production ML models
Required skills
Infrastructure-as-code (IaC), LLM serving infrastructure, model orchestration, CI/CD pipelines, telemetry and observability, incident response, prompt governance, model versioning, security and compliance
Preferred skills
Azure OpenAI, self-hosted OSS models, vector databases, ensemble strategies, A/B experiments, cost optimization
Technologies
Azure OpenAI, self-hosted OSS models, vector databases
Responsibilities
Architect secure reusable IaC frameworks for Gen AI and LLM operations; Design, deploy, and maintain LLM serving infrastructure; Implement model orchestration including routing, ensemble strategies, fallbacks, retries, and cache layers; Build CI/CD pipelines for prompt catalogs, model configurations, guardrails, and evaluation suites; Define and track LLM-specific SLIs (latency, response quality, safety violations, hallucination rate); Implement telemetry (traces, logs, metrics, prompt/response analytics) and A/B experiments; Establish alerting and incident response playbooks; Lead development and standardization of CI/CD pipelines for AI/ML model deployment; Ensure security, privacy, and regulatory compliance; Manage prompt governance including versioning, approval workflows, and rollback; Define and enforce best practices for model versioning governance and lifecycle management; Troubleshoot and resolve issues related to LLM deployment, scaling, and performance; Stay updated with advancements in ML Ops, LLMs, and Gen AI technologies
Seniority
Principal, hands-on IC with strategic oversight
