Senior Engineering Manager, AI

Role overview Lead the team responsible for a high-throughput AI gateway that routes language-model traffic across multiple providers and models. The role combines people leadership, distributed-systems strategy, and close partnership across engineering, product, research, and executive stakeholders to scale the service while balancing availability, latency, cost, and reliability.

Responsibilities - Manage, mentor, and grow a team of up to 10 senior and staff engineers, primarily distributed across North America. - Build a strong team culture centered on technical quality, predictable delivery, and career development. - Set the architecture and scaling strategy for a gateway handling very large volumes of model traffic, including routing, caching, queuing, and fallback behavior. - Guide the integration and evaluation of multiple model providers and dozens of models, balancing latency, cost, and reliability. - Align the platform roadmap with product needs, AI research, and engineering leadership. - Establish engineering standards and telemetry for tracking token usage and costs.

Requirements - Experience leading and developing high-performing teams of senior or staff engineers. - A strong background designing, building, and operating high-throughput, low-latency distributed systems, including routing, caching, queuing, or rate limiting. - Ability to engage deeply in architecture discussions, give clear technical direction, and contribute to design reviews. - Strong communication and partnership skills for aligning stakeholders across an R&D organization. - Active use of AI tools in developer workflows and ability to encourage their effective use across a team.

Nice to have - Applied AI experience or familiarity with frameworks such as LangChain, LangGraph, LiteLLM, or OpenRouter. - Experience with workflow orchestrators or proxies such as Temporal or Envoy. - Familiarity with Python, Java, and distributed caching or queuing systems.

Benefits and work setup The listing mentions potential equity and a benefits program.