AI Model Policy Trainer, Generalist (Remote)

Role overview

Evaluate how AI systems respond to user requests by applying detailed safety and policy frameworks. You will interpret nuanced conversations, make consistent judgments about model behavior, and explain decisions clearly, especially when small contextual differences change the appropriate classification. The work is remote and involves regular exposure to sensitive subject matter.

Responsibilities

- Learn and apply new policies, taxonomies, definitions, rubrics, and decision standards across varied projects. - Review user prompts, model responses, and relevant conversation history in full context. - Distinguish between closely related policy categories, severity levels, and borderline cases. - Choose the most defensible classification for ambiguous examples and write concise, evidence-based rationales. - Identify unclear guidance, contradictions, policy gaps, and emerging edge cases, then raise well-reasoned questions. - Participate in calibration discussions, incorporate feedback, and help improve evaluation examples, decision rules, and quality standards.

Requirements

- Strong analytical judgment and the ability to make precise distinctions between similar cases. - Clear, concise written communication supported by relevant evidence and policy language. - Ability to interpret both the wording and intent of a policy without substituting personal beliefs for the stated standard. - Careful, consistent performance during repetitive, feedback-heavy evaluation work. - Ability to learn unfamiliar subject matter quickly, handle incomplete or conflicting context, and work independently. - Maturity and sound judgment when reviewing difficult or potentially disturbing material.

Nice to have

- Experience in quality assurance, research, editing, law, teaching, operations, trust and safety, content moderation, compliance, investigations, or policy work. - Professional experience evaluating language-model outputs or working in AI evaluation, model quality, data annotation, or reinforcement-learning feedback. - Familiarity with detailed rubrics, taxonomies, editorial standards, quality audits, calibration, inter-rater agreement, or adjudication. - Experience creating policy guidance, decision trees, evaluation examples, or structured rationales.

Benefits and work setup

- Remote role for applicants in the United States; California residents are not eligible. - W-2 employment with an indicated compensation range of $35-$55. - Monday through Friday schedule, generally 8:00 AM to 5:00 PM Pacific Time. - Work may involve content concerning sexual material, emotional distress, self-harm, suicide, violence, weapons, abuse, exploitation, and discrimination.