Quality Evaluator – Fluent English (Europe & LATAM)

Role overview Evaluate responses from AI models to help improve their accuracy, usefulness, and reliability. You will compare answers to the same prompt, judge how well they meet the user’s needs, and explain your decisions using consistent criteria.

Responsibilities - Work with different AI models and tools across varied prompts and topics. - Compare two or more generated responses side by side. - Assess accuracy, helpfulness, instruction-following, clarity, and overall quality. - Apply detailed rating guidance and multi-step evaluation workflows consistently. - Write clear, well-reasoned explanations for evaluation decisions.

Requirements - Fluent English, with strong reading comprehension and writing skills. - Extensive hands-on experience using large language models and critically assessing their outputs. - Strong analytical judgment when weighing multiple aspects of response quality. - Careful attention to detail and ability to follow complex instructions. - Ability to explain decisions clearly and work across different tools and workflows.

Nice to have - Experience in AI response evaluation, data annotation, or quality assurance. - Background in editing, research, fact-checking, or analytical writing. - Familiarity with evaluation rubrics and side-by-side comparisons.

Benefits and work setup - Remote, part-time independent contract work for up to three months. - Compensation of $7–$12 per hour.