AI Benchmark Engineer | Native Language Specialist - Spanish (Argentina) - Remote

Role overview Design and validate multilingual software engineering benchmarks that stress-test large language models in non-English terminal workflows. This remote freelance engagement is aimed at native-speaking software engineers who can author realistic, locale-faithful tasks without leaning on English translations. The work directly shapes how rigorously AI coding agents are evaluated on prompt-language robustness, encoding edge cases, and locale-specific behaviour.

Responsibilities - Engineer benchmark tasks for terminal-based coding agents, including realistic file and dataset assets kept in the target language. - Identify failure points where current models stumble on multilingual prompts, non-English inputs, and locale-aware processing. - Build reference implementations and write deterministic verifier scripts, resorting to rubric-based judging only when unavoidable. - Calibrate task difficulty from Easy to Very Hard by running standard terminal-bench configurations against small, medium, and large model tiers. - Participate in a four-layer human quality-control process, paired with automated checks, to guarantee grammatical accuracy and benchmark integrity. - Analyze execution logs to refine difficulty, fairness, and reproducibility across the suite.

Requirements - 5+ years of professional software engineering experience, ideally with a background at leading technology firms or top-tier engineering universities. - Native or near-native fluency in Spanish (Argentina), with strong command of grammar, register, and phrasing, plus high English proficiency. - Strong proficiency in Python, standard shell scripting, and data processing. - Extensive day-to-day experience with terminal and CLI-based development, including hands-on familiarity with coding agents. - Deep technical understanding of multilingual text-processing pitfalls, including Unicode normalization, encoding/decoding robustness, locale-dependent collation, casing and non-Gregorian dates, text I/O, and safe string operations.

Nice to have - Familiarity with right-to-left rendering, typography, or font fallbacks for adjacent languages.

Benefits and work setup - Fully remote independent-contractor engagement with flexible hours; set your own schedule and workload. - Competitive freelance rates with prompt payment and no invoice chasing. - Engagement structured as 1099 work, so contractors handle their own taxes and benefits; geographic restrictions may apply where international sanctions or embargoes are in force. - Best suited as supplemental, project-based income rather than a full-time role, since availability fluctuates with project demand.