Research Engineer - Pretraining

Role overview Take model pretraining work from early architecture experiments through large-scale training runs, with a target scale of up to 1–3 trillion parameters. The role combines research decisions with hands-on engineering across the model and training stack, including solving systems issues that affect whether a run completes successfully.

Responsibilities - Build and extend pretraining code, moving promising small experiments into full training runs. - Evaluate architecture choices, including mixture-of-experts and attention designs, against model quality and training cost. - Profile training jobs and improve end-to-end throughput in collaboration with systems engineers. - Make long training runs more resilient and address issues that threaten successful completion. - Design and run experiments, then explain what the results establish and what remains uncertain.

Requirements - Experience with model pretraining or large-scale machine learning research engineering. - Ability to work across model architecture and training systems concerns. - Strong experimental judgment and clear communication of results and limitations. - Good judgment about when to investigate independently and when to involve colleagues.

Benefits and work setup - Cash compensation of $150,000–$350,000, including equity incentives. - Flexible remote or in-person work in San Francisco. - Visa sponsorship and relocation assistance may be available. - Quarterly team gatherings, hackathons, conferences, and learning opportunities.