AI Benchmark Engineer | Native Language Specialist - Chinese (Taiwan) - Remote
Core
Design, build, and validate rigorous evaluation benchmarks (Terminal-Bench) to test large language models on multilingual software challenges, specifically focusing on non-English data processing and locale/encoding edge cases.
Role type
Native language specialist / AI benchmark engineer
Builds
High-signal multilingual benchmark tasks and deterministic verifier scripts for coding agents
Domain
AI evaluation / Multilingual software engineering
Deliverable
production ML models
Required skills
Python, shell scripting, data processing, terminal/CLI workflows, coding agents, multilingual text processing (encoding/decoding, Unicode, locale conventions)
Preferred skills
Experience at leading tech companies, top-tier engineering university graduation, deep understanding of language grammar/register/phrasing
Technologies
Python, shell scripting, Terminal-Bench, Haiku, Sonnet, Opus
Responsibilities
Create realistic task environments using native language datasets, find failure points in AI models within native language contexts, write reliable verifier scripts, analyze execution logs to calibrate task difficulty, participate in 4-layer human quality control process
Seniority
Mid-level, hands-on IC
