AI Benchmark Engineer | Native Language Specialist - Spanish (Mexico) - Remote
Core
Design, build, and validate rigorous evaluation benchmarks (Terminal-Bench) to test large language models on multilingual software challenges, specifically focusing on non-English data processing and locale/encoding edge cases.
Role type
Freelance AI Benchmark Engineer (Native Language Specialist)
Builds
High-signal evaluation tasks and deterministic verifier scripts for multilingual coding agents.
Domain
Artificial Intelligence / Large Language Models / Multilingual Software Engineering
Deliverable
production ML models
Required skills
Python, shell scripting, data processing, Terminal/CLI development, coding agents, multilingual text processing, Unicode normalization, locale-dependent conventions
Preferred skills
Experience at leading technology companies, graduation from top-tier engineering universities, deep understanding of language grammar and register
Technologies
Python, shell scripting, Terminal-Bench, Haiku, Sonnet, Opus
Responsibilities
Create realistic task environments using native language datasets, find failure points in AI models within native language contexts, write reliable verifier scripts, analyze execution logs to calibrate task difficulty, participate in a 4-layer human quality control process.
Seniority
Mid-level, hands-on IC
