Software Engineer, Trust & Safety

Role overview

This is a hands-on individual contributor position on a Trust & Safety team at an AI routing and infrastructure platform that processes millions of large language model requests and a large volume of payments every day. The engineer will own the detection, enforcement, and review systems that keep abuse, fraud, malicious users, and illegal content off the platform. Work spans the inference path, the payments path, internal review tooling, and the underlying data platform, with little process between ideas and shipping.

Responsibilities

- Build and operate detection systems across signup, payments, and usage surfaces to identify abuse and fraud early. - Own the technical enforcement pipeline end-to-end, from detection proposing a candidate through human review to restrictions landing across systems. - Develop internal investigation and review tooling, including case queues, evidence summaries, bulk review and enactment, and alerting mechanisms. - Ship content safety solutions on the inference path, covering illegal content detection and reporting, along with KYC and external intelligence integrations. - Investigate incidents directly in the data and build the analytics plane so the team can quickly size patterns and separate real abuse from false-positive clusters. - Build defensible monitoring for risk, abuse spikes, and fraud while reducing false positives, and partner with data scientists on risk and abuse ML model features.

Requirements

- 4+ years building and operating production systems, ideally including trust and safety, fraud, payments risk, security, or anti-abuse work. - Proficiency in React, TypeScript, Next.js, and JavaScript runtimes. - Ability to write SQL against large event datasets and reason about base rates, precision and recall, and the cost of a wrong decision. - Sound judgment when working with incomplete evidence and strong written and verbal communication across audiences. - High agency and bias toward action, motivated by adversarial problems where the counterparty adapts. - Comfortable in a small, fast-moving environment with intentionally fluid team boundaries, plus discretion and resilience when handling disturbing content or sensitive user data.

Nice to have

- Experience with payments fraud tooling such as Stripe Radar, chargeback and dispute flows, crypto payment risk, or with identity and KYC systems. - Familiarity with LLM-specific abuse patterns including jailbreaks, prompt injection, key theft and resale, shared or scraped credentials, and automated account farming. - Experience operating large-scale analytical datastores like ClickHouse or BigQuery and observability platforms. - Familiarity with reporting and compliance obligations for hosted AI, including illegal content reporting and model provider policy requirements. - Active side projects or hands-on work in AI products, infrastructure, or developer tooling.