← Back to jobs

Remote job

Clinical AI Quality Specialist

AI Engineer Full-time Permanent United States

Job details

$130,000 Salary
United States Eligibility
Lead Experience
Full-time Employment

About this role

Role overview

Assess the clinical quality and safety of an AI assistant designed to support people managing cardiovascular health. You will evaluate behavior before and after releases, review real user conversations, and work with product and engineering partners to turn clinical findings into clear, trackable decisions.

Responsibilities - Create test cases, including ambiguous and challenging scenarios, and score system responses against expected clinical behavior. - Build and maintain evaluation datasets and contribute to methods for scoring results. - Review sampled production conversations on a weekly basis and help prepare monthly reports that explain the scope and limits of the review. - Assess handling of emergencies, self-harm risk, and medication safety, including whether escalation is excessive or insufficient. - Document failures, likely contributing factors, and clinical risk in terms engineering can act on; ensure findings have an owner, ticket, and decision. - Maintain current, auditable clinical specifications, review records, and supporting evidence.

Requirements - PharmD or equivalent clinical doctorate, an active U.S. license, and at least three years of combined clinical practice and clinical AI evaluation. - Experience evaluating large language model systems, reviewing production conversations, and working with monitoring tools or similar workflows. - Strong knowledge of triage, escalation, crisis and self-harm response, and medication safety based on current guidance. - Ability to investigate assumptions, consult and cite guidelines, drug labels, and primary sources, and verify AI-generated material before relying on it. - Clear written and verbal communication, including the ability to explain clinical risk to product and engineering teams and maintain precise documentation. - Comfort with ambiguity, feedback, and evolving AI evaluation practices.

Nice to have

Experience in cardiovascular care or hypertension management, structured clinical transcript rating and rater agreement, crisis or self-harm review, or use of monitoring tools such as Langfuse.

Benefits and work setup

The stated U.S. base salary range is $130,000–$143,000 annually. The posting says this figure covers salary and bonus, and excludes equity and benefits.

Skills detected in the listing

LLM
Detected Nov 21, 2026
Last verified Nov 22, 2026

Hidden Jobs Access

Unlock application links

Read the full job details for free. An active Hidden Jobs Access subscription is required to open the original application link.

Weekly

$6.99 /week
  • Original application links
  • Full access to all MCP features
  • Cancel anytime

Monthly

$35.99 $17.99 /month
  • Original application links
  • Full access to all MCP features
  • 35% cheaper than weekly

Lifetime

$99.99 $49.99 /forever
  • Original application links
  • Full access to all MCP features
  • One-time payment
Hidden Jobs gives subscribers direct access to original application links