← Back to jobs

Remote job

Staff AI Platform Engineer

AI Engineer Full-time Place, Boston

Job details

Not specified Salary
Place, Boston Eligibility
Staff Experience
Full-time Employment

About this role

Role overview Set technical direction and build production foundations that let engineering teams develop and operate AI-enabled software reliably. This staff-level engineering role spans model inference, gateways, agent orchestration, context services, observability, and experimentation, with an emphasis on robust systems and close collaboration with AI researchers.

Responsibilities - Define platform architecture and technical direction, write design documents and RFCs, guide build-versus-buy choices, and align stakeholders. - Deploy, benchmark, and tune scalable inference for open-weight models using runtimes such as vLLM, SGLang, or TensorRT-LLM. - Build and operate a model gateway for self-hosted and commercial models, including consistent authentication, routing, failover, quotas, and cost tracking. - Create reusable agent orchestration components and context services for memory, retrieval, and task-specific data discovery. - Instrument services with traces and metrics, operate them against service objectives, and build systems for experiment tracking and reproducible results. - Design APIs, security, and deployment approaches with potential external use and customer environments in mind; communicate platform plans with product and business teams.

Requirements - Typically 8+ years of software engineering experience, including 4+ years building and operating production AI, ML, or LLM systems, with staff-level scope. - Strong production Python and platform engineering skills across APIs, distributed systems, containers, Kubernetes, CI/CD, and testing. - Experience shipping agentic systems and improving their reliability, plus context engineering with retrieval, embeddings, search, or agent memory. - Experience instrumenting and operating services against objectives such as SLOs. - Working knowledge of transformers, LLM inference, experimentation, benchmarking, and model evaluation; familiarity with PyTorch and Hugging Face and experience fine-tuning, evaluating, or serving language models. - Evidence of technical leadership across systems or teams, including architecture ownership, design writing, roadmap development, and engineer mentorship.

Nice to have - Experience building an LLM gateway or API gateway with routing, authentication, quotas, rate limits, failover, and cost attribution. - GPU inference optimization, including profiling or techniques such as caching, speculative decoding, or parallelism. - Experience with evaluation and experiment-management tools, productizing internal platforms, or deploying AI in customer-controlled or regulated environments.

Benefits and work setup Medical, dental, and vision premiums are fully covered, with a retirement plan offering 5% matching, uncapped vacation plus sick leave and public holidays, flexible hybrid or remote work, and relocation assistance for qualifying employees. Compensation depends on experience and location.

Skills detected in the listing

PythonStakeholder ManagementKubernetesLLM
Detected Nov 17, 2026
Last verified Nov 17, 2026

Hidden Jobs Access

Unlock application links

Read the full job details for free. An active Hidden Jobs Access subscription is required to open the original application link.

Weekly

$6.99 /week
  • Original application links
  • Full access to all MCP features
  • Cancel anytime

Monthly

$35.99 $17.99 /month
  • Original application links
  • Full access to all MCP features
  • 35% cheaper than weekly

Lifetime

$99.99 $49.99 /forever
  • Original application links
  • Full access to all MCP features
  • One-time payment
Hidden Jobs gives subscribers direct access to original application links