AI Benchmark Engineer | Native Language Specialist - Chinese (Hong Kong) - Remote

Role overview A remote, freelance opportunity for native Chinese-speaking software engineers to design multilingual evaluation benchmarks for large language models. The work centers on building rigorous, verifiable Terminal-Bench tasks that probe how AI coding agents handle non-English software environments, locale-dependent data, and complex encoding edge cases.

Responsibilities - Design and engineer Terminal-Bench tasks that evaluate coding agents in Chinese-language scenarios - Build realistic task environments using Chinese-language datasets and files, keeping assets in the target language - Identify failure points where AI behavior degrades in Chinese-only contexts - Develop reference implementations and write deterministic verifier scripts, using rubric-based judging only when strictly necessary - Calibrate task difficulty by analyzing execution logs and running benchmarks against multiple model tiers - Participate in a multi-layer human quality control process alongside automated LLM-based checks

Requirements - 5+ years of professional software engineering experience - Background at leading technology companies and/or a top-tier engineering university - Native or near-native Chinese fluency with strong English proficiency - Strong Python and standard shell scripting skills - Extensive Terminal/CLI development experience and familiarity with coding agents - Deep understanding of multilingual text processing pitfalls, including Unicode normalization, locale-dependent conventions (collation, casing, non-Gregorian dates), text I/O, toolchain interoperability, and safe string operations

Nice to have - Bidirectional/RTL handling, font fallbacks, and rendering or typography considerations (for applicable languages)

Benefits and work setup - Independent contractor (1099) position with flexible scheduling and no fixed hours - Competitive rates with prompt payment - Fully remote work - Best treated as supplemental income rather than a full-time role; no health insurance, paid time off, or retirement benefits - Geographic restrictions may apply for regions under international embargo or sanctions - Contractors are responsible for their own tax obligations