Senior AI Engineer

Role overview

Build the analytical layer that evaluates how engineers work with AI tools. This role designs the scoring rubrics, underlying models, and data pipelines that transform raw session traces into clear signals that an engineering leader can act on. It is the role most central to defining how the product measures value.

Responsibilities

- Design rubrics and evaluation models that score how effectively engineers use AI in their work - Build pipelines that process raw session traces into structured, actionable scoring outputs - Prototype and ship the systems that power core insights for a VP-level engineering audience - Iterate on evaluation quality, calibration, and reliability as new use cases surface - Partner with engineering leadership to ensure outputs map to decisions they actually make - Contribute to the overall architecture of the analytics and scoring layer

Requirements

- Senior-level experience as an AI or ML engineer building production evaluation or scoring systems - Strong background working with language models, prompt design, or model-based evaluation - Ability to design rubrics, metrics, or scoring frameworks for complex human behavior - Comfort building data pipelines that handle structured and unstructured inputs - Care for evaluation rigor, calibration, and producing outputs that leaders can trust and act on - Product-minded engineering instincts, with comfort shipping iteratively and refining over time

Benefits and work setup

- Central role in defining what the product is and how it measures value - Direct partnership with senior engineering leadership