RLHF & LLM Alignment Scientist

Role overview Develop evaluation and alignment methods for advanced language models, from preference data collection through reward model validation and adversarial testing. The work combines research, benchmark design, automated analysis, and coordination with subject matter experts who create nuanced training data.

Responsibilities - Build evaluation rubrics, pairwise ranking approaches, and prompt sets for models handling complex reasoning. - Develop automated checks for hallucinations, reward hacking, and sycophantic responses. - Coordinate specialist contributors producing synthetic data in areas such as programming, medicine, and law. - Contribute research on alignment methods and the efficiency of data curation.

Requirements - Practical experience with language model evaluation benchmarks, such as MMLU, GSM8K, or pairwise arena rankings. - Strong understanding of RLHF, DPO, PPO, supervised fine-tuning, and prompt engineering. - Experience writing precise technical guidelines and evaluation rubrics.

Nice to have - Publications or notable open-source contributions related to model alignment.

Benefits and work setup - Full-time, senior role, with remote work available within the United Kingdom. - Listed base compensation of £110,000–£145,000 plus equity. - At least four weeks of paid time off, with additional leave available, including up to 16 weeks of paid parental leave. - Opportunity to work directly with frontier AI researchers and model labs.