CareerPlan

Evaluation Engineer, Applied AI

ParisFull-time2026-10-19 → 2026-11-20

Core

Build frontier LLM evaluation benchmarks and frameworks to assess model capabilities in real-world industry use cases (e.g., CAD, finance) and guide product development.

Role type

Senior IC evaluation engineer (LLM)

Builds

Industry-specific LLM benchmarks, evaluation agents, and data pipelines

Domain

Artificial Intelligence / Large Language Models / Enterprise Software

Deliverable

production ML models

Required skills

Python, LLM training and inference, benchmark design, data sourcing and cleaning, stakeholder workshops, technical writing

Preferred skills

AI evaluation experience, synthetic data generation, open-source contributions, domain expertise (engineering/finance/science)

Technologies

Python, LLM frameworks

Responsibilities

Run discovery workshops with customers and experts to define use cases; Design and build benchmarks where agents operate inside real software; Stress-test existing evals for flaws; Source data or build synthetic proxies; Present results to senior leaders; Help teams build reusable evaluation tools

Seniority

Senior, hands-on IC