← Back to jobs

Remote job

Senior Research Engineer, AI Evaluation

AI Engineer Full-time Permanent United States

Job details

$180,000 Salary
United States Eligibility
Senior Experience
Full-time Employment

About this role

Role overview Lead the design and implementation of evaluation systems for language models used in clinical decision support applications. The role combines applied research and engineering to measure model reasoning, retrieval, and structured responses, and to make quality and safety checks reproducible across development and deployment.

Responsibilities - Build and maintain automated evaluation frameworks for model outputs. - Develop general and domain-specific benchmarks, synthetic test cases, and trusted reference datasets. - Measure and address edge cases, hallucinations, model drift, and safety regressions. - Establish offline and online evaluation practices with research, infrastructure, and product partners, including integration into CI/CD workflows.

Requirements - At least three years in software engineering, applied machine learning, or AI research, with strong statistics and scientific rigor. - Demonstrated experience building LLM evaluation harnesses or benchmarks, such as model-based judging, reference metrics, or human review. - Advanced Python skills and experience with PyTorch or Hugging Face, model APIs, retrieval-augmented generation, and experiment tracking or observability tools. - Strong technical writing and communication skills.

Nice to have - Advanced degree in computer science, machine learning, statistics, or another scientific field. - Peer-reviewed research or presentations at machine learning conferences. - Knowledge graph experience.

Benefits and work setup - Base salary range of $180,000–$240,000, equity, health and wellness coverage, and paid time off. - Flexible work arrangement with a Lower Manhattan hub and support for remote work.

Skills detected in the listing

PythonMachine LearningLLM
Detected Nov 18, 2026
Last verified Nov 19, 2026

Hidden Jobs Access

Unlock application links

Read the full job details for free. An active Hidden Jobs Access subscription is required to open the original application link.

Weekly

$6.99 /week
  • Original application links
  • Full access to all MCP features
  • Cancel anytime

Monthly

$35.99 $17.99 /month
  • Original application links
  • Full access to all MCP features
  • 35% cheaper than weekly

Lifetime

$99.99 $49.99 /forever
  • Original application links
  • Full access to all MCP features
  • One-time payment
Hidden Jobs gives subscribers direct access to original application links