Staff Site Reliability Engineer - Dev Platform

Role overview Lead reliability engineering for an internal language-model gateway used by engineering teams to access multiple model providers. The role combines production operations, platform architecture, and technical leadership: you’ll help the service remain dependable as usage grows while balancing resilience, security, cost, and developer needs.

Responsibilities - Own service availability and performance, including service-level objectives, error budgets, and on-call practices. - Design routing, retries, rate limits, failover, and graceful degradation for upstream provider disruptions. - Set direction for gateway architecture, capacity planning, and scaling. - Build observability for latency, errors, token usage, and cost, with useful attribution by team and workload. - Work with security, finance, and governance partners on access controls, budgets, audit logs, and data handling. - Lead incident response and blameless reviews, turn findings into reliability improvements, and mentor engineers. - Automate infrastructure and operations through code, safe deployments, and progressive releases.

Requirements - Extensive experience operating business-critical production services in site reliability, infrastructure, or platform engineering. - Hands-on experience with Kubernetes, a major cloud platform, and infrastructure-as-code tools such as Terraform. - Strong practical knowledge of reliability objectives, observability, and incident management. - Proficiency in Rust or Python, with the ability to work in a production codebase. - Experience with API gateways, reverse proxies, or traffic management, including load balancing, rate limiting, and failover. - Familiarity with Consul and Vault, and with API authentication and authorization such as OAuth/OIDC, JWT, mutual TLS, or API keys. - Staff-level technical leadership: setting direction across teams, writing design documents, and bringing others along.

Nice to have - Experience with LLM gateways or provider APIs, model evaluation pipelines, coding agents in sandboxed environments, or agent memory systems and their databases. - Experience serving models with vLLM or running distributed inference and training with Ray. - Kubernetes controller or operator development, usage-based service cost management, or work in security-sensitive or regulated settings. - Contributions to open-source gateway, proxy, or observability projects.