← Back to jobs

Remote job

Member of Technical Staff, Evals

AI Engineer Full-time San Francisco or Remote

Job details

Not specified Salary
San Francisco or Remote Eligibility
Staff Experience
Full-time Employment

About this role

Role overview Develop evaluation methods that help teams understand the capabilities and performance of machine learning models. The work spans model training and deployment, combining benchmark design, dataset analysis, and careful investigation of whether results reflect the behaviors they claim to measure. Accuracy, reproducibility, and clear interpretation are central to the role.

Responsibilities - Design and validate evaluations and benchmarks for models across training and deployment. - Investigate model behavior and identify areas that current evaluation methods miss. - Analyze dataset quality and run data ablations to find opportunities to improve training data. - Audit benchmarks, research papers, and open-source evaluation implementations for errors or flawed assumptions. - Define metrics that meaningfully reflect model quality and improve the reliability of measurements. - Build tools that make evaluation results accessible and useful to teams working on models and products.

Requirements - Experience developing model evaluations, benchmarks, or machine learning experiments. - Strong programming skills and experience working with machine learning systems. - A record of owning technical projects from planning through completion. - Strong understanding of post-training techniques and how they affect model performance. - Experience analyzing results, with a skeptical and evidence-based approach to findings that cannot be reproduced or explained. - Comfort with ambiguity and a commitment to rigorous, reliable evaluation.

Benefits and work setup The role can be based in San Francisco or remotely. The listed US annual salary range is $225,000–$550,000, depending on experience, with equity as a significant part of compensation. Benefits include a 401(k) plan with 6% salary matching, health, dental, and vision coverage for employees and dependents, and unlimited paid time off. Visa sponsorship and relocation support are available for candidates moving to San Francisco.

Skills detected in the listing

Machine Learning
Detected Nov 19, 2026
Last verified Nov 19, 2026

Hidden Jobs Access

Unlock application links

Read the full job details for free. An active Hidden Jobs Access subscription is required to open the original application link.

Weekly

$6.99 /week
  • Original application links
  • Full access to all MCP features
  • Cancel anytime

Monthly

$35.99 $17.99 /month
  • Original application links
  • Full access to all MCP features
  • 35% cheaper than weekly

Lifetime

$99.99 $49.99 /forever
  • Original application links
  • Full access to all MCP features
  • One-time payment
Hidden Jobs gives subscribers direct access to original application links