Remote job
Senior Software Engineer, AI Training - US
Job details
About this role
Role overview Assess the quality of interactions between developers and AI coding agents. This contract role focuses on whether an agent’s responses show sound engineering judgment, communicate clearly, and help users make progress; it does not involve building production software.
Responsibilities - Review coding-agent interactions from start to finish, judging usefulness and high-level correctness. - Evaluate explanations, reasoning, and guidance alongside the proposed code. - Apply consistent quality distinctions and explain what separates stronger responses from weaker ones. - Write direct, detailed feedback about what worked, what was confusing, or what could undermine user trust. - Help define standards for effective AI-assisted development workflows.
Requirements - Senior-level software engineering experience, such as Senior, Staff, Principal, or equivalent. - Strong experience with TypeScript or JavaScript, or Python. - Hands-on use of at least one modern AI coding assistant and familiarity with AI-assisted development workflows. - Ability to assess code and engineering decisions without running every example or inspecting every line. - Strong written and spoken English (at least B2), including the ability to write detailed evaluations and record short video explanations. - Confidence making rigorous, clearly explained judgments and giving candid feedback.
Nice to have - Exposure to prompt design or model evaluation. - Experience mentoring senior engineers or setting engineering standards.
Benefits and work setup - Remote contract engagement, with flexible hours of roughly 10–20 per week; location eligibility may vary. - Hourly rate is listed as $100–$200, depending on experience and location. - Work is ongoing, with individual projects typically lasting from a few weeks to a few months; further projects may be offered based on performance. - Selection involves a take-home evaluation and recorded walkthrough. Work begins when the exercise is passed and a project has an opening.