Remote job
Applied AI Engineer
Job details
About this role
Role overview
Build and ship AI-powered capabilities for a business communications product, with responsibility spanning agent workflows, evaluation, and the infrastructure that supports them. The work is hands-on and iterative: you’ll partner with senior engineers and product colleagues to move ideas into production, measure whether they work, and keep them dependable in real use.
Responsibilities
- Design and implement agent workflows, including routing, tool use, and feedback loops. - Build evaluation datasets, automated grading, and regression checks; use offline and live measures to assess model and prompt changes. - Turn production failures into new evaluations that help catch similar issues earlier. - Develop and operate AI infrastructure with platform and reliability engineers, including deployment paths for workloads on Kubernetes. - Track model costs, latency, and failures; design graceful behavior for timeouts, rate limits, and unreliable outputs, and respond to production issues. - Manage prompts and model settings as versioned changes that can be reviewed, released, and rolled back safely. - Help protect regional data boundaries and reduce risks such as unintended data exposure and prompt injection.
Requirements
- Bachelor’s degree or higher; the available source truncates the degree field of study, so it is not specified here. - Experience shipping LLM features in production and working across application code and infrastructure. - Ability to build reliable, measurable AI systems and use evaluation results to guide iteration. - Comfort partnering with product, engineering, and reliability teams, making progress on defined problems, and communicating blockers and learnings.
Nice to have
Experience with the listed production environment: Ruby on Rails, Vue.js, Amazon Bedrock, AWS and EKS, Terraform or Terragrunt, Sidekiq, MySQL or PostgreSQL, Argo CD, and Karpenter.
Benefits and work setup
This is a new position based in downtown Toronto, with the team working in the office three days per week or as needed. The listed salary range is $140,000–$160,000.