Senior AI Engineer - Model Selection and Orchestration

Role overview

This is a senior engineering role focused on building production-grade AI systems, with particular emphasis on the decision and orchestration layer that determines how AI tasks should be executed. The work spans customer-facing AI experiences and internal agent workflows in a financial technology setting, where reliability, security, and sound engineering judgment are fundamental. The position is remote-first for US-based employees, with the option to work in-person from a Manhattan office.

Responsibilities

- Design and implement the routing layer that decides whether a request should be handled by deterministic logic, retrieval, a classifier, a smaller or reasoning model, a tool call, or human escalation. - Build intent classification and clarification flows that distinguish information requests from actions, evaluate confidence, and identify ambiguous input. - Benchmark rules, retrieval approaches, and models against representative tasks, using quality, latency, safety, and cost metrics to guide selection. - Implement model adapters, structured outputs, timeouts, retry strategies, and fallback behavior in a testable orchestration layer that avoids over-reliance on any single model provider. - Connect agents to approved APIs and tools with appropriate authentication, authorization, policy checks, and customer confirmation, handling duplicate requests, idempotency, and recovery for state-changing actions. - Partner with security and risk functions to implement input and output protections, sensitive-data handling, prompt-injection defenses, and controlled access. - Develop reusable orchestration patterns, adapters, and routing components that make it easier for other engineering teams to build AI-powered experiences. - Measure and improve AI behavior in production through structured evaluation work.

Requirements

- 10+ years of professional experience building, testing, deploying, and operating production software, including hands-on delivery of AI or ML-enabled applications beyond prototypes. - Strong proficiency in Python or TypeScript, with modern software engineering practices including API design, automated testing, distributed services, and debugging across application, model, and tool boundaries. - Practical experience building applications using LLM tool calling, structured outputs, retrieval-augmented generation, and agent or workflow orchestration. - Ability to evaluate classification quality, uncertainty, fallback strategies, and different model or system approaches using measurable outcomes such as task success, latency, reliability, and total inference cost. - Experience with cloud deployment, automated testing, CI/CD, observability, identity and access controls, secrets management, retries, idempotency, and failure recovery in production systems. - Strong judgment about when a problem should be solved with deterministic software, retrieval, a model, or a combination, and when an agent should ask for clarification or hand work to a person.

Benefits and work setup

- Remote-first role for US-based employees, with the option to work in-person from a Manhattan office. - Flexible working culture and incentive programs. - Unlimited paid time off, generous paid parental leave, and family support policies. - Company-sponsored retirement plan with employer match. - Learning and wellness subscription stipend. - Employer-sponsored insurance for employees and dependents, with several fully covered options. - Reported annual base pay range of $162,000 to $205,000, with additional premium percentages possible based on a tiered premium strategy; bonus and equity may also be part of the total compensation package.