Remote job
Research Engineering Manager, STEM
Job details
About this role
Role overview Lead the research engineering team building synthetic STEM data and evaluations for training and assessing advanced models. Partner closely with research scientists to shape experiments and technical direction, while staying hands-on with code, data quality, and evaluation design. Hiring and developing the team are central parts of the role.
Responsibilities - Set team priorities, hire and coach research engineers, and establish clear project ownership and review practices. - Define standards for evaluation methods, data quality, and reproducibility so datasets and results can be regenerated reliably. - Partner with scientists to turn research hypotheses into experiments with clear baselines, ablations, and decision criteria. - Build scalable pipelines to generate, verify, filter, deduplicate, and version synthetic data; measure how its characteristics affect model performance. - Develop benchmarks for under-tested capabilities, with calibrated difficulty, contamination safeguards, and sound statistical methods. - Create quality controls such as expert-reviewed reference sets, calibrated model-based grading, audits, and agreement measures.
Requirements - At least eight years of hands-on machine learning or research engineering experience, including five or more in industry and three or more managing technical staff. - Production ownership of synthetic or curated data pipelines used to guide training decisions, with evidence of measured quality and downstream impact. - Experience designing and shipping evaluations or benchmarks used to make model decisions; understand contamination, saturation, grader bias, agreement, variance, and confidence intervals. - Strong experimental judgment, including choosing credible baselines and planning ablations to distinguish real effects from noise. - Strong Python skills and production experience with PyTorch or JAX; working knowledge of post-training methods such as supervised fine-tuning, preference optimization, and reinforcement learning. - Experience setting technical direction and scaling distributed data or model workloads on cloud infrastructure.
Nice to have Experience at a frontier AI research lab, or familiarity with distributed processing tools such as Ray, Spark, or Kubernetes.