Remote job
Founding Software Engineer — AI & Voice Systems
Job details
About this role
Role overview A founding-level engineering position focused on building the evaluation and measurement infrastructure for an AI voice calling platform. The work centers on scoring production calls in near real time, layering audio-native signals (pronunciation, emotion, vocal stress) on top of LLM-based evaluators, and powering a simulation engine that places realistic automated calls. Getting this measurement layer right is treated as foundational to everything else in the product.
Responsibilities - Design and operate evaluation pipelines that score large volumes of production calls at low latency. - Build audio-native metrics that go beyond transcripts, validating them against human-labeled ground truth alongside LLM-based evaluators. - Extend a simulation engine that dials real agents across phone and WebRTC, modeling varied personas, accents, and background noise. - Own large-scale analytics infrastructure, including columnar data stores, key-value streams, and the query layer feeding internal dashboards. - Orchestrate long-running, distributed workflows (replays, backfills, load tests) using a durable workflow engine, with strong attention to idempotency and correctness. - Operate what you build: dashboards, alerts, and root-cause investigation when production metrics drift.
Requirements - Four or more years of production experience building backend, ML, or audio systems, ideally in TypeScript or Python. - Hands-on depth in at least one of: speech/audio processing, LLM evaluation pipelines, or high-volume data infrastructure. - Strong rigor around correctness, with comfort thinking in terms of idempotency, replay safety, and ground-truth labels rather than only happy paths. - Willingness to own operations, including dashboards, alerts, and after-hours investigation. - Clear written communication, with design documents and post-mortems people actually want to read.
Nice to have - Familiarity with the listed stack (TypeScript, Bun, AWS via SST, ClickHouse, DynamoDB, Temporal, Postgres, and WebRTC/telephony integrations). - Experience validating ML systems against human judgment at scale.