Staff AI Engineer, Agents and Evals

Role overview Shape the behavior of an AI agent that helps people design and make physical objects, and establish the evaluations that determine whether changes are ready to ship. You will work across context, memory, retrieval, model selection, and tool use, partnering with engineering to make the agent reliable and useful from request through manufactured result.

Responsibilities - Manage the agent’s context, retrieval, and memory as a shared budget, including decisions about what information to surface and when. - Define dependable tool interfaces, including input schemas, results, and failure behavior. - Build evaluation suites, live scoring practices, and regression gates for changes to agent behavior. - Improve how the agent reasons, asks questions, explains its work, and recovers from failures. - Select models for different tasks and define the evidence a new model must meet before release. - Communicate evaluation findings in reports that support product and engineering decisions.

Requirements - Demonstrated experience operating agent systems in production, including tool use, multi-step workflows, streaming, and recovery. - Experience designing graded evaluations, regression checks, and human review processes. - Practical context engineering across prompts, retrieval, and memory, with attention to measured cost. - Ability to design model-usable tool contracts, including schemas, outputs, and error handling. - Python experience, plus either work in a substantial production codebase with a strong test baseline or a published benchmark you built. - Clear technical writing for decision makers.

Nice to have - Experience with production retrieval systems, published tool-use benchmarks, or voice and UX writing for agent products.

Benefits and work setup - Remote within the United States, with occasional travel to Detroit. - Base pay of $170,000–$220,000, equity with four-year vesting and a one-year cliff, and health, dental, and vision coverage. - US work authorization is required; visa sponsorship is unavailable. The work may involve data subject to export-control requirements.