← Back to jobs

Remote job

Software Engineer, Evaluation Tooling

Full Stack Full-time United States

Job details

Not specified Salary
United States Eligibility
Not specified Experience
Full-time Employment

About this role

Role overview Build internal software that runs evaluations of AI assistants and language models at scale. The work covers querying systems, preserving the context needed to reproduce their outputs, and helping research and analysis teams compare results across models and versions.

Responsibilities - Build services that query AI assistants and language models through APIs and other interfaces. - Create data pipelines that store outputs with enough context to reproduce each run. - Develop tools for comparing answers across models, versions, and time. - Build internal interfaces for running, reviewing, and annotating evaluations. - Keep tools reliable, tested, secure, and well documented.

Requirements - Professional experience building production software in TypeScript or Python. - Experience with data pipelines, APIs, and relational databases such as PostgreSQL. - Familiarity with language model APIs and practical constraints such as rate limits, cost, and nondeterministic outputs. - Care for correctness, testing, and reproducibility.

Benefits and work setup - Full-time role based in Scottsdale, Arizona, or remote within the United States. - Annual pay range of $140,000–$220,000. - Health insurance and paid time off.

Skills detected in the listing

TypeScriptPythonPostgreSQL
Detected Nov 18, 2026
Last verified Nov 18, 2026

Hidden Jobs Access

Unlock application links

Read the full job details for free. An active Hidden Jobs Access subscription is required to open the original application link.

Weekly

$6.99 /week
  • Original application links
  • Full access to all MCP features
  • Cancel anytime

Monthly

$35.99 $17.99 /month
  • Original application links
  • Full access to all MCP features
  • 35% cheaper than weekly

Lifetime

$99.99 $49.99 /forever
  • Original application links
  • Full access to all MCP features
  • One-time payment
Hidden Jobs gives subscribers direct access to original application links