Remote job
Rust - Senior Engineer - AI Interaction Evaluator
Job details
About this role
Role overview A contracted, part-time role evaluating the quality of interactions produced by AI coding agents, with a focus on whether those interactions reflect the judgment of a senior Rust engineer. Rather than shipping production code, the evaluator assesses how the model reasons, explains, and guides a developer through real-world Rust scenarios. It is well suited to senior systems engineers who enjoy critiquing craft and setting the standard for AI-assisted tooling.
Responsibilities - Review AI-generated coding interactions end-to-end, weighing usefulness, high-level correctness, and alignment with senior-level engineering judgment in Rust. - Judge the quality of explanations and reasoning, not just the code itself. - Calibrate ratings by articulating what separates a poor response from a strong one. - Provide opinionated feedback describing what worked, what missed, and what felt off or misleading. - Contribute to defining what an excellent interaction looks like inside AI-first IDEs and editor workflows.
Requirements - Senior- or staff-level engineering background with deep, current experience writing Rust in production contexts. - Hands-on familiarity with modern AI coding tools such as OpenAI Codex, Claude Code, and Cursor-class editors. - Ability to evaluate code and conversation quality without executing or auditing every line in detail. - Comfort making rigorous subjective calls about engineering taste and the quality of technical communication. - High bar for what strong Rust engineering looks like in practice.
Nice to have - Prior exposure to prompt design, evaluation rubrics, or annotation workflows. - Experience mentoring senior engineers or establishing engineering standards inside a team.
Benefits and work setup - Contract engagement running through early May, with potential extension. - Approximately 10 to 20 hours per week, remote and async-friendly. - Hourly rate between $100 and $200, with an ASAP start. - Hiring process consists of a take-home evaluation exercise plus one behavioral interview.