CareerPlan

Senior AI Engineer (f/m/d) - Remote in Germany

Remote jobRemoteFull-time€77k–€97k2026-09-29 → 2026-11-19
Based in Germany

Core

Design, build, and evaluate agentic AI systems in production, focusing on creating evaluation frameworks (golden datasets, LLM-as-judge pipelines) to ensure agent reliability and trustworthiness.

Role type

Senior AI Engineer (Agentic Systems & Evaluation)

Builds

Agentic systems, evaluation frameworks, golden datasets, and CI/CD integration for agent regression testing.

Domain

Artificial Intelligence / Agentic Systems / LLM Evaluation

Deliverable

production ML models

Required skills

Agent graph design (LangGraph), LLM evaluation pipeline design, Python (FastAPI, PostgreSQL), LLM API integration (Anthropic, OpenAI, Azure, Bedrock, Vertex AI), Tracing tools (Langfuse, LangSmith), Reliability engineering (retries, error handling)

Preferred skills

Experience with supervisor/sub-agent patterns, Statistical reasoning for evaluation (CIs, sample sizes)

Technologies

LangGraph, FastAPI, PostgreSQL, Langfuse, LangSmith, Anthropic, OpenAI, Azure Foundry, Bedrock, Vertex AI

Responsibilities

Build and extend agentic systems including new tools and sub-agents; Develop and maintain evaluation frameworks with golden datasets and LLM-as-judge pipelines; Integrate evaluation regressions into CI/CD to block PRs; Collaborate with R&D to automate engineering work based on customer insights.

Seniority

Senior, hands-on IC