AI Engineer

Role overview Improve the reliability of AI agents that handle real customer conversations for businesses. You’ll work across prompts, tool use, information retrieval, and evaluation, using measured results to address common failure patterns and create improvements that benefit agents across the platform.

Responsibilities - Build evaluation datasets and pipelines that reveal how agents perform in realistic conversations. - Improve prompts, tool calling, and retrieval to help agents remain accurate and focused. - Compare models on response quality, speed, and cost, and inform model choices. - Identify recurring failure modes and address them through platform-wide improvements. - Partner with product and design on how agents gather information, recommend services, and hand off conversations to people.

Requirements - Hands-on experience shipping LLM features to real users. - Experience creating evaluations and using their results to guide decisions. - Strong Python or TypeScript skills. - A measurement-led approach to assessing changes, including before-and-after comparisons. - Clear written communication about experiments, outcomes, and lessons learned.

Nice to have - Experience with agent frameworks or multi-step tool use. - Experience with voice agents or customer-facing chat products.

Benefits and work setup Remote contract role with flexible hours. Compensation starts at $5,500 per month.

Application exercise Build and publicly publish an agent for a real business using the provided free model. Create at least 20 realistic test conversations, including difficult cases; record failures, adjust the agent’s instructions, knowledge, or workflow, then rerun the same tests. Submit the agent link, test set, and a short account of the before-and-after results. There is no stated time limit.