Research Staff, LLMs

Role overview Research and develop large language models, from early experiments to models ready for production. The work spans data, model architecture, training methods, and distributed systems, with an emphasis on finding practical approaches to difficult problems in AI.

Responsibilities - Collaborate with researchers to identify and shape new LLM research questions. - Survey recent research, compare methods, and distill findings into directions for further work. - Design and run experiments to evaluate ideas and improve model performance. - Train transformer models on distributed compute infrastructure and help deploy resulting models. - Present findings and technical concepts clearly to relevant audiences. - Track developments in deep learning and assess their potential applications.

Requirements - At least three years of applied deep learning research experience, including knowledge of neural network architectures and loss functions. - Hands-on experience with LLMs, including data curation, large-scale distributed training, transformer optimization, and reinforcement learning. - Strong Python programming skills and experience with PyTorch. - Familiarity with multiple transformer designs, such as autoregressive and sequence-to-sequence models. - Experience with distributed computing, large-scale data processing, and experimental model optimization. - Ability to communicate complex technical ideas and analyze results in depth.

Nice to have - Deep knowledge of transformer and causal language model architectures. - Experience with distributed LLM training or inference and RLHF data labeling or training pipelines. - Relevant deep learning publications and familiarity with recent LLM methods. - An interest in building new systems and using AI to automate or amplify research work.