LLM Platform Engineer
Core
Build and maintain evaluation, observability, and internal tooling systems for LLM products and AI workflows.
Role type
Senior LLM Platform Engineer
Builds
Evaluation and observability systems, internal self-service tools, data pipelines, and RAG-based repositories for AI products.
Domain
Artificial Intelligence / Large Language Models / Data Engineering
Deliverable
production ML models
Required skills
Python, SQL, LLM evaluation pipelines, observability systems, data pipeline design, web application development (Vue/React/TypeScript), RAG architecture, CI/CD integration, schema evolution.
Preferred skills
Business analytics, data visualization, LLM evaluation frameworks, GitHub Actions, monitoring tools.
Technologies
Python, SQL, Vue, React, TypeScript, Model Context Protocol, GitHub Actions.
Responsibilities
Partner with engineers and product managers to identify workflow bottlenecks and own solutions from requirements to rollout. Own evaluation and observability systems for AI products including automated and human review. Build internal applications for inspecting AI results and investigating failures. Create unified analytics layers from evaluation results and telemetry. Design RAG-based information repositories. Integrate AI insights with CI/CD workflows. Mentor teammates and champion code quality and reliability.
Seniority
Senior, hands-on IC
