AI QA Trainer - LLM Evaluation - Freelance Project

Role overview Evaluate the accuracy, safety, and reliability of language models through structured testing and detailed failure analysis. The work focuses on LLM evaluation, including test design, adversarial checks, regression coverage, and quality reporting, to help teams find and address weaknesses in model behavior.

Responsibilities - Test model responses against realistic prompts and scenarios, checking factual accuracy, reasoning, and grounding. - Design test plans, regression suites, scoring rubrics, and clear pass/fail criteria. - Evaluate risks such as hallucinations, prompt injection, jailbreaks, bias, unsafe outputs, incorrect tool use, and retrieval errors. - Document reproducible failures, propose root-cause hypotheses, and report findings clearly. - Recommend improvements to prompts, safeguards, and evaluation measures such as precision, recall, faithfulness, toxicity, or latency. - Contribute to red-team exercises, test automation, and dashboards that track quality over time.

Requirements - Strong understanding of AI or language-model quality assurance, model evaluation, safety testing, or a related discipline. - Clear, evidence-based communication, including the ability to explain evaluation decisions and reproduce issues. - A degree in computer science, data science, computational linguistics, statistics, or a related field is preferred. - Experience shipping QA for AI/ML systems, designing rubrics, or conducting adversarial testing is valuable. - Python or SQL test automation experience is useful; familiarity with tools such as PyTest or LLM evaluation frameworks is a plus. - Knowledge of grounding checks, fairness audits, regression testing, and retrieval-augmented systems is beneficial.

Benefits and work setup - Remote, contract work. - The listed hourly range is $6–$65; the final rate depends on experience, expertise, and location. - Contractors supply a secure computer and high-speed internet. Employer-sponsored health insurance and paid time off are not included.