AI Inference Engineer QVAC (100% remote Worldwide)

Role overview Build and maintain the C++ inference layer that enables machine learning models to run efficiently on users’ edge devices. The work focuses on runtime quality: improving startup, memory use, throughput, latency, hardware compatibility, and stability over long sessions. You will help turn research capabilities into dependable, private on-device AI features that can operate without relying on cloud inference.

Responsibilities - Deploy machine learning models to edge devices using inference frameworks such as llama.cpp and ggml. - Adapt and optimize inference engines for different device and GPU architectures. - Work with researchers to implement ideas and move models from research into production. - Integrate machine learning capabilities into existing products. - Evolve the runtime abstractions that support new inference features while maintaining performance and reliability. - Investigate runtime behavior, including model loading, memory pressure, latency, throughput, and long-session operation.

Requirements - Strong C++ programming skills and experience with llama.cpp and ggml inference engines. - Experience with at least one GPU framework: CUDA, Vulkan, Metal, or OpenCL. - Good understanding of deep learning concepts, model architectures, transformers, large language models, and diffusion models. - Ability to learn new technologies and techniques quickly. - A degree in computer science, AI, machine learning, or a related field, with a track record in AI research and development.

Nice to have - Experience training or fine-tuning large language models, or putting models into production. - Research experience with new model architectures. - Distributed systems experience or JavaScript skills.

Benefits and work setup The position is described as fully remote worldwide. It involves work with a globally distributed team and focuses on building local AI infrastructure for peer-to-peer products.