Remote job
Staff AI Platform Engineer
Job details
About this role
Role overview Set technical direction and build production foundations that let engineering teams develop and operate AI-enabled software reliably. This staff-level engineering role spans model inference, gateways, agent orchestration, context services, observability, and experimentation, with an emphasis on robust systems and close collaboration with AI researchers.
Responsibilities - Define platform architecture and technical direction, write design documents and RFCs, guide build-versus-buy choices, and align stakeholders. - Deploy, benchmark, and tune scalable inference for open-weight models using runtimes such as vLLM, SGLang, or TensorRT-LLM. - Build and operate a model gateway for self-hosted and commercial models, including consistent authentication, routing, failover, quotas, and cost tracking. - Create reusable agent orchestration components and context services for memory, retrieval, and task-specific data discovery. - Instrument services with traces and metrics, operate them against service objectives, and build systems for experiment tracking and reproducible results. - Design APIs, security, and deployment approaches with potential external use and customer environments in mind; communicate platform plans with product and business teams.
Requirements - Typically 8+ years of software engineering experience, including 4+ years building and operating production AI, ML, or LLM systems, with staff-level scope. - Strong production Python and platform engineering skills across APIs, distributed systems, containers, Kubernetes, CI/CD, and testing. - Experience shipping agentic systems and improving their reliability, plus context engineering with retrieval, embeddings, search, or agent memory. - Experience instrumenting and operating services against objectives such as SLOs. - Working knowledge of transformers, LLM inference, experimentation, benchmarking, and model evaluation; familiarity with PyTorch and Hugging Face and experience fine-tuning, evaluating, or serving language models. - Evidence of technical leadership across systems or teams, including architecture ownership, design writing, roadmap development, and engineer mentorship.
Nice to have - Experience building an LLM gateway or API gateway with routing, authentication, quotas, rate limits, failover, and cost attribution. - GPU inference optimization, including profiling or techniques such as caching, speculative decoding, or parallelism. - Experience with evaluation and experiment-management tools, productizing internal platforms, or deploying AI in customer-controlled or regulated environments.
Benefits and work setup Medical, dental, and vision premiums are fully covered, with a retirement plan offering 5% matching, uncapped vacation plus sick leave and public holidays, flexible hybrid or remote work, and relocation assistance for qualifying employees. Compensation depends on experience and location.