Head of Evaluations (Legal AI Benchmarking)

Role overview Lead the design and growth of evaluation methods for AI tools used in legal work. You will build benchmarks and datasets, assess model outputs, and work with engineers to turn evaluation findings into improvements in legal reasoning, factual accuracy, and safe response behavior.

Responsibilities - Create benchmarks for contract drafting and review, legal research, and information extraction. - Source, build, and maintain datasets for evaluating legal AI performance. - Review and score generated legal text, contract analyses, and interpretations of statutes for accuracy and precision. - Set evaluation criteria for reasoning quality, citation accuracy, and appropriate abstention when a model should not answer. - Work with engineering teams to translate identified errors into actionable guidance for model improvement.

Requirements - A master’s or doctoral degree in statistics, mathematics, machine learning, or a related field, or equivalent preparation. - Strong statistical and analytical skills. - Ability to structure complex statutes and case law into clear, logical data points. - Familiarity with Python and statistical tools such as pandas, NumPy, and Jupyter notebooks. - Authorization to work in the applicable country without employer-sponsored visa support.

Benefits and work setup - Remote role open to candidates in the United States, United Kingdom, or Norway. - Work in a fast-changing environment where priorities and processes may evolve; the role calls for initiative and comfort creating structure as needed.