Trust & Safety Lead, Education

Role overview Lead trust and safety for an education platform where educators create AI applications for classroom use. The role covers both AI behavior and user-created content, with responsibility for incident decisions, moderation policy, and the systems that help detect harm. You will balance hands-on response with proactive testing and work closely with product, engineering, learning, privacy, and school-facing teams.

Responsibilities - Review flagged material, investigate emerging misuse, and test features through policy-breaking scenarios and red teaming. - Define a harm taxonomy and labeling guidance, then create labeled examples that engineering can use to develop and assess detection systems. - Specify what automated filters and signals should catch, evaluate their performance, and track false alarms and detection gaps. - Lead incident handling from triage and investigation through action, school communication, required reporting, and post-incident review. - Maintain moderation policies, review new features for safety risks, and establish clear escalation paths and operational guidance. - Equip school and learning partners with reporting processes and training, and share incident patterns and metrics with leadership and product teams.

Requirements - Experience making trust and safety decisions, including difficult cases involving young people, such as self-harm, abuse, or inappropriate contact. - Experience designing detection operations: labeling rules, filter requirements, and ways to determine whether controls work. - Ability to write practical policies, runbooks, triage tools, training materials, and operational dashboards. - Calm judgment under pressure, with the ability to act decisively, explain reasoning, and escalate appropriately. - Strong communication with school leaders, engineers, and partner teams in a remote, asynchronous setting.

Nice to have - Generative AI testing, red teaming, evaluations, adversarial testing, or proactive abuse investigation. - Experience with user-generated content platforms, creator tools, or marketplace integrity. - Familiarity with student privacy and child safety rules, or experience in schools, counseling, or education technology. - Experience building trust and safety operations from the ground up.

Benefits and work setup Remote-first, with autonomy and close collaboration in a small mission-focused team. The listing describes healthcare, dental and vision coverage, retirement benefits, life and AD&D coverage, a wellness stipend, a home office budget, a laptop, and generous time off.