Director, Research - Evaluation & Training

Role overview Lead a research team focused on evaluating AI models, analyzing their failures, and measuring how training and evaluation data affect performance. The team’s findings will guide which benchmarks and datasets to develop, help demonstrate the value of data, and inform product, market, and customer decisions.

Responsibilities - Set and own a multi-quarter research roadmap for model evaluation, error analysis, and data valuation. - Analyze model failures and benchmark results to identify capability gaps and emerging trends. - Turn evaluation findings into recommendations about datasets and benchmarks to prioritize. - Develop methods to measure how data contributes to model performance. - Lead, grow, and support a research team, setting expectations for rigor, quality, and execution. - Communicate findings to product and commercial teams and customers, connecting technical evidence to their needs.

Requirements - At least seven years of experience in applied AI, machine learning, or research, including at least four years managing technical teams. - A record of translating research and analysis into outcomes relevant to customers or the business. - Strong judgment about AI and machine learning markets and the ability to prioritize amid a changing competitive landscape. - Working depth in language-model evaluation, benchmarking, and model behavior analysis to set direction and assess experimental quality. - Ability to identify meaningful signals across evaluation results and failure cases. - Excellent communication and storytelling skills, including explaining technical findings to non-research audiences.

Nice to have - Familiarity with data valuation or data attribution research. - Experience with frontier AI labs, public benchmarks, or commercial AI data and evaluation products.

Benefits and work setup - Listed annual salary range: $275,000–$425,000 USD. Compensation depends on qualifications, experience, skills, and location.