Research Engineer - Inference

Role overview Build inference systems that support reinforcement learning research, with a focus on scaling to very large models across multi-node GPU clusters. The work aims to increase rollout throughput and make frequent model updates practical, including for long-horizon agent tasks. This is a full-time role based in San Francisco or remote.

Responsibilities - Extend an inference engine integration to support additional model architectures in reinforcement learning training, validating changes at small scale before larger runs. - Increase rollout throughput for agents with long contexts by tuning concurrency and reuse of the key-value cache. - Improve multi-node inference for mixture-of-experts models, including expert parallelism and prefill/decode execution choices. - Reduce the overhead of transferring updated model weights between training and inference systems. - Diagnose performance bottlenecks and validate optimizations through experiments.

Requirements - Strong experience with machine learning systems and model inference. - Ability to work with distributed GPU workloads and optimize their performance. - Experience evaluating system changes through careful testing and measurement.

Nice to have - Contributions to vLLM or another inference engine. - CUDA or Triton kernel development, especially for low-precision inference.

Benefits and work setup - Cash compensation of $150,000–$350,000, including equity incentives. - Flexible remote or in-person work in San Francisco. - Visa sponsorship and relocation assistance may be available. - Quarterly team gatherings, hackathons, conferences, and learning opportunities.