AI Engineer - Guardrails and Observability
Core
Engineering and operating third-party guardrail and observability applications for GenAI (LLM/RAG) systems on an on-premises OpenShift platform.
Role type
Senior IC GenAI operations engineer (guardrails & observability)
Builds
Production guardrail and observability pipelines for enterprise GenAI applications
Domain
Financial services / Generative AI operations
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Python, OpenShift/Kubernetes, LLM/RAG application deployment, AI guardrails, model evaluation workflows, API integration, orchestration frameworks (LangChain), observability dashboards
Preferred skills
Galileo, LangSmith, Splunk, LlamaIndex, Haystack, agentic workflow evaluation, annotation pipelines, fine-tuning, prompt lifecycle management
Technologies
OpenShift, Kubernetes, LangChain, Galileo, LangSmith, Splunk, OpenAI APIs
Responsibilities
Develop, deploy, and operate vendor applications in an on-premises OpenShift environment; Build and maintain APIs connecting guardrail platforms to GenAI apps; Monitor application performance, safety, and guardrail metrics; Implement and maintain model-based evaluation workflows; Build observability dashboards and alerting solutions; Troubleshoot production issues and optimize performance
Seniority
Senior, hands-on IC with leadership responsibilities