AI Benchmark Engineer | Native Language Specialist - Spanish (Mexico) - Remote
Core
Design, build, and validate rigorous evaluation benchmarks (Terminal-Bench) to test large language models on multilingual software challenges and terminal workflows.
Role type
Native language specialist / AI benchmark engineer (contractor)
Builds
Multilingual evaluation tasks, realistic task environments, and verification scripts for coding agents.
Domain
AI evaluation, multilingual software engineering, terminal/CLI workflows
Deliverable
production ML models
Required skills
Python, shell scripting, data processing, terminal/CLI development, coding agents, multilingual text processing, Unicode normalization, locale conventions
Preferred skills
Experience at leading tech companies, top-tier engineering university background, native Spanish fluency, high English proficiency
Technologies
Python, shell scripting, Terminal-Bench, Haiku, Sonnet, Opus
Responsibilities
Create high-signal tasks in native language, find AI failure points in native language, write deterministic verifier scripts, analyze execution logs and calibrate task difficulty, participate in 4-layer human quality control process.
Seniority
Mid-level, hands-on contractor
