Staff / Principal Rust Engineer, AI Coding Agent Evaluator

Role overview

This is a contract position focused on evaluating AI coding agents rather than writing production software. The work centers on judging whether modern AI assistants respond like seasoned engineers — reasoning correctly, explaining clearly, and guiding users well. It's a part-time engagement (roughly 10–20 hours per week) suited to a Staff or Principal-level engineer who can make rigorous subjective judgments about interaction quality.

Responsibilities

- Review end-to-end exchanges between developers and AI coding agents, judging usefulness, plausibility, and overall quality - Assess reasoning, preamble, and explanation quality rather than line-by-line code correctness - Calibrate response quality across tiers, articulating what separates a mediocre answer from a strong one - Provide opinionated written feedback describing what worked, what missed, and what felt misleading - Contribute to defining the standard for what a great AI-assisted development experience should feel like

Requirements

- Staff or Principal engineering background (or equivalent industry experience) - Strong fluency in TypeScript/JavaScript or Python - Hands-on experience with AI coding tools such as OpenAI Codex, Claude Code, or Cursor - Comfort evaluating code and interactions without executing or auditing every line - Ability to deliver direct, well-reasoned critique with a high bar for engineering quality - Familiarity with modern AI-assisted development workflows

Nice to have

- Prior experience with AI-first IDEs and agentic coding environments - Exposure to prompt design, evaluation rubrics, or model review workflows - Background mentoring senior engineers or establishing engineering standards

Benefits and work setup

- Hourly rate of $100–$200 depending on experience - Part-time schedule of approximately 10–20 hours per week - Contract running through early May, with possible extension - Fully remote asynchronous work - Hiring process consists of a take-home evaluation and a single behavioral interview