Remote job
AI Platform Engineer
Job details
About this role
Role overview Build and operate production features powered by large language models. The work spans practical AI delivery, from prompt and retrieval design through live monitoring and quality assurance, with a focus on dependable behavior under real traffic.
Responsibilities - Design prompts and retrieval approaches for LLM-backed product features. - Add evaluation loops to measure output quality and catch regressions. - Implement guardrails and fallback behavior for cases where model responses are unsuitable or unavailable. - Monitor production behavior and use observability signals to identify quality or reliability issues. - Perform quality checks as features move from development into live use.
Requirements - Experience delivering software features that use large language models in production or production-like systems. - Ability to design and assess prompts and retrieval workflows. - Familiarity with evaluation methods for model output and regression testing. - Understanding of guardrails, fallback strategies, and live-system observability.