AI Engineer - Monitoring, Observability, and Governance
Core
Engineering and operating third-party applications in an on-premises OpenShift environment to support enterprise GenAI monitoring, observability, and governance.
Role type
Senior IC AI Engineer (GenAI Operations & Governance)
Builds
Guardrails, observability platforms, and model-based evaluation workflows for LLM- and RAG-powered systems
Domain
Financial services / Generative AI Operations
Deliverable
production ML models | dashboards & analysis | infrastructure
Required skills
Python, OpenShift/Kubernetes, LLM/RAG production deployment, API development, AI guardrails, model-based evaluation, LangChain, observability, inference optimization
Preferred skills
Galileo, LangSmith, Splunk, LlamaIndex, Haystack, agentic AI workflows, annotation pipelines, prompt versioning
Technologies
OpenShift, Kubernetes, LangChain, Galileo, LangSmith, Splunk, OpenAI APIs
Responsibilities
Develop and support vendor applications in on-premises OpenShift; Build APIs integrating guardrails and observability tools; Implement monitoring for performance, safety, and quality metrics; Calibrate model-based evaluation workflows; Create observability dashboards and alerting; Troubleshoot production issues and improve scalability; Lead and mentor team members
Seniority
Senior, hands-on IC with technical leadership
