← Back to jobs

Remote job

Staff Data Scientist, AI Evaluation

Data Scientist Full-time United States

Job details

Not specified Salary
United States Eligibility
Staff Experience
Full-time Employment

About this role

Role overview

Develop rigorous ways to measure and improve the quality and performance of AI systems. This hands-on data science role spans applied AI, engineering, and product analytics: you will work with production data, design experiments, investigate system behavior, and turn findings into decisions and improvements.

Responsibilities - Create quantitative evaluation methods and tools for assessing AI system quality and performance. - Design and run experiments that compare approaches, reveal tradeoffs, and measure the effects of changes. - Establish repeatable monitoring to detect regressions, anomalies, and shifts in system performance. - Extract, transform, and analyze production data for technical and product investigations. - Diagnose issues involving data, models, and production systems, then identify likely causes and possible remedies. - Partner with Engineering, Product, and Analytics to communicate findings and guide improvements.

Requirements - Strong data science skills for evaluating complex systems and interpreting results. - Experience designing experiments and quantitative measures of quality or performance. - Ability to work with production data, including extracting, transforming, and analyzing it. - Skill in investigating problems across data, models, and deployed systems. - Clear communication and ability to collaborate across technical and product teams.

Skills detected in the listing

PythonSQLStakeholder ManagementMachine Learning
Detected Nov 18, 2026
Last verified Not yet verified

Hidden Jobs Access

Unlock application links

Read the full job details for free. An active Hidden Jobs Access subscription is required to open the original application link.

Weekly

$6.99 /week
  • Original application links
  • Full access to all MCP features
  • Cancel anytime

Monthly

$35.99 $17.99 /month
  • Original application links
  • Full access to all MCP features
  • 35% cheaper than weekly

Lifetime

$99.99 $49.99 /forever
  • Original application links
  • Full access to all MCP features
  • One-time payment
Hidden Jobs gives subscribers direct access to original application links