Remote job
Staff AI Engineer, Agentic Systems and Evaluation
Job details
About this role
Role overview Design and own the core agent infrastructure used in client production systems, along with the evaluation and tracing needed to check that it remains reliable over time. This is a hands-on architecture role with a short path from design to deployment and close feedback from the engineers operating the systems.
Responsibilities - Build orchestration, state and memory, and tool interfaces for agentic systems. - Design cost controls for systems that rely on models and tools. - Develop evaluation and tracing approaches that provide evidence of system reliability. - Make it possible to detect and investigate changes in behavior after launch, including model updates and shifts in client data. - Support the engineers deploying and debugging these systems in production.
Requirements - Strong engineering capability in agentic systems and their underlying infrastructure. - Experience designing orchestration, state or memory, and tool interfaces. - Ability to build evaluations and tracing that help assess reliability after deployment. - Focus on production behavior over time, including the effects of changing models and data. - Willingness to work closely with engineers who deploy and support the systems.