Product Engineer

Role overview

Build and operate the software used to evaluate AI agents, run them in sandboxed environments, and present results through web consoles and agent-facing resources. The role is primarily backend and Python-focused, with work spanning scoring, infrastructure, interfaces, and browser-based experiences.

Responsibilities

- Develop and maintain a scoring engine and the services that run agents in controlled environments. - Build and support consoles used to inspect agent runs and evaluation results. - Work across APIs, data models, migrations, and storage using Python, FastAPI, SQLAlchemy, SQLModel, and Alembic. - Integrate browser automation and agent protocols, including Playwright and MCP components. - Deploy and operate services using containers and cloud infrastructure, including Redis and object storage. - Improve reliability across complete agent workflows, including error handling, receipts, and edge cases.

Requirements

- Ability to reproduce results consistently and support conclusions with repeatable evidence. - Clear technical writing for readers across engineering, risk, and marketing. - Comfort working within regulated processes, approved environments, and audit requirements. - Attention to operational details and recovery paths, including the less visible final stages of a user journey.

Nice to have

- Experience with Python 3.11, FastAPI, PostgreSQL, SQLite, Docker, and cloud container services. - Familiarity with Jinja, vanilla JavaScript and CSS, Astro, React, TypeScript, Node.js, the Anthropic SDK, or MCP.

Application process

Applicants can complete an interview through their own AI agent, which records the candidate’s own responses to a set of questions. The agent may clarify questions or help correct spelling, but must not write answers. Candidates who prefer not to use an agent can apply by email through the address provided in the original listing.