AI Benchmark Engineer | Native Language Specialist - Dutch - Remote

Role overview Create and validate terminal-based benchmark tasks that measure how well coding models handle software challenges in Dutch. The work is freelance and remote, with an emphasis on realistic target-language datasets and edge cases that expose weaknesses in multilingual text handling rather than tasks that can be solved by translating everything into English.

Responsibilities - Design software engineering tasks for evaluating coding agents in terminal workflows. - Build realistic task environments with datasets and files written in Dutch. - Identify failure cases involving language-specific prompts and multilingual data processing. - Develop reference solutions and deterministic verifier scripts; use rubric-based evaluation only when necessary. - Run benchmark configurations, review execution logs and calibrate task difficulty from easy to very hard. - Participate in human review and audit stages, checking fairness, grammar, reliability and benchmark integrity.

Requirements - At least five years of professional software engineering experience. - Native or near-native Dutch proficiency, including command of grammar, register and phrasing, plus strong English. - Strong Python, shell scripting and data-processing skills. - Extensive terminal and command-line development experience, with working familiarity with coding agents. - Deep understanding of multilingual text issues, including encoding, Unicode normalization, locale-specific sorting and casing, dates, text input/output and safe string handling.

Benefits and work setup - Remote independent contracting with flexible scheduling and project-based availability. Work volume may fluctuate, so the opportunity may suit supplemental income better than a primary full-time role. - Contractors handle their own taxes and do not receive employee benefits such as health insurance, paid time off or retirement contributions. Geographic restrictions may apply.