← Back to jobs

Remote job

ML Systems Engineer, On-Device Inference

AI Engineer Full-time San Francisco or remote

Job details

Not specified Salary
San Francisco or remote Eligibility
Not specified Experience
Full-time Employment

About this role

Role overview This full-time systems engineering role builds inference software that helps language and voice models run effectively on phones, personal computers, and other devices. The work spans profiling, execution-path optimization, and collaboration with model researchers to translate architectural choices into responsive and sustained performance on real hardware. The role is listed as based in San Francisco or remote.

Responsibilities - Profile model loading, prompt processing, decoding, and streamed output across mobile devices and desktop hardware. - Optimize kernels, execution graphs, memory allocation, and cache use for target processors. - Develop mixed-precision and quantized execution paths, checking numerical correctness and model quality as well as speed. - Measure sustained performance across realistic context lengths, concurrent workloads, and thermal conditions. - Build repeatable benchmarks and regression checks for latency, memory use, energy consumption, and reliability. - Collaborate on model and runtime design, and package improvements in maintainable interfaces for other engineers.

Requirements - Strong systems programming skills in C++, Rust, or a comparable language, plus Python for experiments and tooling. - Understanding of transformer inference, memory hierarchies, and limits on compute or bandwidth utilization. - Experience with inference runtimes, hardware acceleration, or machine-learning deployment on constrained devices. - Ability to use profiling and measurements to identify bottlenecks and verify improvements. - Careful debugging and testing practices, including attention to numerical changes and performance regressions.

Nice to have Experience with Metal, MLX, Core ML, CUDA, mobile GPU or NPU execution, low-bit quantization, custom kernels, compiler optimization, or streaming audio systems.

Skills detected in the listing

JavaScriptPythonGoRustC++
Detected Nov 17, 2026
Last verified Nov 17, 2026

Hidden Jobs Access

Unlock application links

Read the full job details for free. An active Hidden Jobs Access subscription is required to open the original application link.

Weekly

$6.99 /week
  • Original application links
  • Full access to all MCP features
  • Cancel anytime

Monthly

$35.99 $17.99 /month
  • Original application links
  • Full access to all MCP features
  • 35% cheaper than weekly

Lifetime

$99.99 $49.99 /forever
  • Original application links
  • Full access to all MCP features
  • One-time payment
Hidden Jobs gives subscribers direct access to original application links