Quality Evaluator – Fluent English (Brazil)

Role overview

Evaluate AI-generated answers and help improve their accuracy, usefulness, clarity, and reliability. You will compare responses across models and topics, apply rating guidance consistently, and explain your judgments in writing.

Responsibilities - Review responses produced by different AI models and tools across a range of prompts. - Compare two or more answers side by side and decide which better meets the request. - Assess accuracy, helpfulness, instruction-following, clarity, and overall quality. - Follow detailed rating criteria and multi-step workflows consistently. - Write concise, well-supported explanations for evaluation decisions.

Requirements - Fluent English, with strong reading comprehension and writing skills. - Extensive practical experience using large language models and critically assessing their outputs. - Strong analytical judgment and attention to detail. - Ability to follow complex instructions and explain decisions clearly. - Comfort switching between AI tools and multi-step tasks.

Nice to have - Background in AI evaluation, data annotation, quality assurance, editing, research, fact-checking, or analytical writing. - Familiarity with evaluation rubrics and side-by-side response comparisons.

Benefits and work setup - Remote, part-time independent contractor engagement for up to three months. - Compensation: $7–$12 per hour.