CareerPlan

AI Benchmark Engineer | Native Language Specialist - Spanish (Mexico) - Remote

Mexico (Remote)RemoteFull-time2026-09-08 → 2026-11-20
RemoteNo visa sponsorship

Core

Design, build, and validate rigorous evaluation benchmarks (Terminal-Bench) to test large language models on multilingual software challenges, specifically focusing on non-English data processing and locale/encoding edge cases.

Role type

Freelance AI Benchmark Engineer (Native Language Specialist)

Builds

High-signal evaluation tasks and deterministic verifier scripts for multilingual coding agents.

Domain

Artificial Intelligence / Large Language Models / Multilingual Software Engineering

Deliverable

production ML models

Required skills

Python, shell scripting, data processing, Terminal/CLI development, coding agents, multilingual text processing, Unicode normalization, locale-dependent conventions

Preferred skills

Experience at leading technology companies, graduation from top-tier engineering universities, deep understanding of language grammar and register

Technologies

Python, shell scripting, Terminal-Bench, Haiku, Sonnet, Opus

Responsibilities

Create realistic task environments using native language datasets, find failure points in AI models within native language contexts, write reliable verifier scripts, analyze execution logs to calibrate task difficulty, participate in a 4-layer human quality control process.

Seniority

Mid-level, hands-on IC