Senior Director, Technical Product Management — AI Quality & Evaluation

Role overview Set product direction for AI quality and evaluation in an early-stage product group, turning frontier AI capabilities into dependable customer experiences. This is a hands-on individual contributor leadership role: you’ll shape strategy and evaluation standards, investigate customer problems, and personally guide a focused feature from discovery through delivery. You’ll work closely with researchers, applied scientists, and engineers, who own production architecture and implementation.

Responsibilities - Create a roadmap that connects customer needs, technical opportunities, and measurable outcomes. - Define shared quality measures, evaluation coverage, and release criteria with research, engineering, and product partners. - Investigate customer failures, define expected behavior, run experiments, and validate whether changes improve outcomes. - Apply sound evaluation practices to non-deterministic AI, including representative coverage, independent validation, and checks for overfitting. - Lead a customer-facing capability that helps people understand AI performance and turn failed interactions into evaluation cases. - Build a feedback and learning process across customer input, evaluation evidence, experiments, and releases. - Coach product partners, designers, and program managers in problem framing, experimentation, and delivery.

Requirements - Track record of building AI/ML products and delivering measurable customer outcomes with increasing ownership. - Deep hands-on experience building AI agents and evaluation capabilities alongside AI research, applied science, or engineering teams. - Strong understanding of AI/ML fundamentals and evaluation reliability. - Product leadership skills, including prioritization, customer investigation, experimentation, and clear communication of tradeoffs. - Comfort inspecting interaction traces and data, prototyping with coding agents, and working directly with customers.

Benefits and work setup - Benefits include medical, dental, and vision coverage; time off; mental health support; paid parental leave; life and disability insurance; 401(k); and employee stock purchase plan.