← Back to jobs

Remote job

Staff Software Engineer, AI Inference

Backend Full-time United States

Job details

Not specified Salary
United States Eligibility
Staff Experience
Full-time Employment

About this role

Role overview

Lead the design and development of a production inference platform for serving large language models. This hands-on staff engineering role sets the technical direction for model serving and runtime optimization while building systems that make inference fast, reliable, cost-conscious, and adaptable as models and infrastructure evolve.

Responsibilities - Define the architecture and roadmap for inference infrastructure, model serving, and runtime optimization. - Build and operate scalable systems for production language model workloads. - Evaluate and integrate inference frameworks, serving runtimes, and related technologies. - Improve latency, throughput, GPU utilization, memory efficiency, and infrastructure costs. - Develop deployment, traffic routing, autoscaling, scheduling, observability, and operations capabilities. - Work with machine learning engineers to productionize models and inference techniques, and establish benchmarks for models, runtimes, and hardware. - Make architectural decisions about building capabilities internally or adopting external solutions.

Requirements - Staff-level software engineering experience designing and operating production systems. - Strong expertise in distributed systems and infrastructure for serving large language models. - Ability to optimize performance across latency, throughput, accelerator use, memory, and cost. - Experience evaluating technical options and making architecture decisions for evolving systems. - Strong collaboration skills across machine learning, infrastructure, and product engineering.

Skills detected in the listing

PythonGoRustC++Machine LearningLLM
Detected Nov 18, 2026
Last verified Not yet verified

Hidden Jobs Access

Unlock application links

Read the full job details for free. An active Hidden Jobs Access subscription is required to open the original application link.

Weekly

$6.99 /week
  • Original application links
  • Full access to all MCP features
  • Cancel anytime

Monthly

$35.99 $17.99 /month
  • Original application links
  • Full access to all MCP features
  • 35% cheaper than weekly

Lifetime

$99.99 $49.99 /forever
  • Original application links
  • Full access to all MCP features
  • One-time payment
Hidden Jobs gives subscribers direct access to original application links