← Back to jobs

Remote job

ML Systems Engineer, On-Device Inference

AI Engineer Full-time San Francisco or remote

Job details

Not specified Salary
San Francisco or remote Eligibility
Not specified Experience
Full-time Employment

About this role

Role overview Build and optimize inference systems that let language and voice models run effectively on phones, personal computers, and other devices. You will find where time, memory, and energy are spent, improve execution on real hardware, and collaborate with model researchers so model choices translate into responsive, sustained performance.

Responsibilities - Profile model loading, prompt processing, decoding, and streaming output across mobile and desktop hardware. - Optimize kernels, execution graphs, memory allocation, and cache behavior for target processors. - Develop mixed-precision and quantized execution paths while checking numerical correctness and model quality. - Measure sustained performance with realistic context sizes, concurrent workloads, and thermal conditions. - Build repeatable benchmarks and regression checks for latency, memory, energy use, and reliability. - Work with research teams on model and runtime design, and package improvements as maintainable interfaces.

Requirements - Strong systems programming in C++, Rust, or a comparable language, plus Python for experiments and tooling. - Understanding of transformer inference, memory hierarchies, and compute or bandwidth bottlenecks. - Experience with inference runtimes, hardware acceleration, or deploying machine learning on constrained devices. - Ability to use profiling and measurements to identify bottlenecks and verify improvements. - Careful debugging and testing, including attention to numerical changes and performance regressions.

Nice to have - Experience with Metal, MLX, Core ML, CUDA, or mobile GPU/NPU execution. - Work on low-bit quantization, custom kernels, compiler optimization, or streaming audio systems.

Benefits and work setup Full-time position; location is San Francisco or remote.

Skills detected in the listing

JavaScriptPythonGoRustC++
Detected Nov 17, 2026
Last verified Nov 17, 2026

Hidden Jobs Access

Unlock application links

Read the full job details for free. An active Hidden Jobs Access subscription is required to open the original application link.

Weekly

$6.99 /week
  • Original application links
  • Full access to all MCP features
  • Cancel anytime

Monthly

$35.99 $17.99 /month
  • Original application links
  • Full access to all MCP features
  • 35% cheaper than weekly

Lifetime

$99.99 $49.99 /forever
  • Original application links
  • Full access to all MCP features
  • One-time payment
Hidden Jobs gives subscribers direct access to original application links