Research Engineer / Scientist, Data & Evals

Role overview Build the data and evaluation systems used to improve AI models for complex legal reasoning and agent workflows. You’ll work closely with researchers and legal experts to turn real-world needs and model failures into reliable training data, useful benchmarks, and measurements that support development decisions.

Responsibilities - Create pipelines that generate, filter, and curate synthetic training and evaluation examples. - Design tests and grading methods that reflect legal work, compare them with expert judgment, and investigate where they fall short. - Develop capability benchmarks and product-focused evaluations based on customer workflows. - Work with domain experts to turn qualitative judgments into rubrics, metrics, and structured data. - Maintain data quality, including coverage, provenance, and protections against evaluation contamination. - Use product feedback and model failure analysis to identify new training and evaluation needs. - Help researchers distinguish meaningful model improvements from misleading metric changes.

Requirements - Experience building data pipelines, machine learning evaluation systems, or experimentation tools. - Strong Python skills and a record of delivering production-quality software. - Sound statistical reasoning and experimental design skills, including awareness of how metrics can mislead. - Ability to turn ambiguous goals or expert feedback into structured datasets and measurable criteria.

Nice to have Experience with synthetic data, language-model evaluations, automated grading, or preference data is useful. Related data and measurement engineering experience close to model development is also relevant. Hiring is open across multiple levels.