CareerPlan

Senior AI Engineer (m/f/n)

Kraków, plFull-time2026-09-17 → 2026-11-19
HybridContractor via own company (B2B)

Core

Building and operating the Evaluations Platform for Von Halsky, InPost's conversational AI shopping assistant, to gate releases and measure LLM performance via automated judging pipelines and eval datasets.

Role type

Senior AI Engineer (Technical Lead)

Builds

LLM-judge pipelines, eval datasets, golden sets, and root-cause analysis tools for a production conversational AI system.

Domain

E-commerce / Conversational AI / LLM Evaluation

Deliverable

production ML models

Required skills

LLM-based system production experience, LLM evaluation (LLM-as-judge, calibration, regression suites), prompt and context engineering, agentic primitives, SQL, cloud infrastructure, Python, AI-assisted development tooling

Preferred skills

Harness and loop engineering, auto-improving systems, adversarial robustness and red-teaming, e-commerce domain experience

Technologies

Python, SQL, LangFuse, Braintrust, Weave, Claude Code, Cursor, Copilot

Responsibilities

Calibrate LLM-judge pipelines against human labels, manage eval datasets and golden sets with coverage across intents, perform root-cause analysis on real conversations, detect abusive or anomalous sessions, define measurable quality definitions for stakeholders, mentor engineers and scope technical work

Seniority

Senior, hands-on IC with technical lead responsibilities