Senior Engineer – AI Monitoring & Control Capabilities
Core
Engineering and operating vendor applications end-to-end on an on-premises OpenShift platform to support enterprise AI monitoring, controls, and governance for LLM-powered systems.
Role type
Senior IC platform engineer (AI operations)
Builds
Production-grade AI guardrail and observability services
Domain
Financial services / AI Agent operations
Deliverable
production ML models
Required skills
Python, OpenShift/Kubernetes, LLM/RAG application deployment, API integration, AI Agent frameworks (OpenAI, LangChain), inference optimization, model evaluation (LLM-as-a-Judge), observability (Grafana)
Preferred skills
Financial services domain experience, annotation pipelines, fine-tuning, prompt versioning, vendor platforms (Galileo, LangSmith, Splunk), LlamaIndex, Haystack
Technologies
OpenShift, Kubernetes, Python, Grafana, LangChain, OpenAI APIs, LlamaIndex, Haystack, Splunk, Galileo, LangSmith
Responsibilities
Own delivery of critical vendor applications from build through production support; Build and maintain APIs connecting guardrail platforms to AI Agent solutions; Establish workflows to monitor performance, safety, and guardrail effectiveness; Implement model evaluation frameworks; Develop observability dashboards and alerting; Troubleshoot production issues and improve scalability; Lead and mentor engineers; Provide input into financial planning and budget tracking.
Seniority
Senior, hands-on IC with mentorship
