Software Engineer, Senior

Role overview Build and optimize software that runs open-source large language models on a specialized inference appliance combining x86 CPUs and FPGAs. The work focuses on modern C++ and performance across the full execution path, from vectorized kernels and memory use to transformer features such as token streaming and KV caching.

Responsibilities - Develop inference software and C++ libraries that use SIMD, threading, and the memory hierarchy efficiently. - Coordinate with FPGA and systems engineers on data movement and computation offload between CPUs and accelerators. - Improve model execution through vectorization, cache-aware techniques, hardware-aware scheduling, and NUMA-aware memory management. - Profile instruction and data-flow bottlenecks and contribute tools and methods for performance analysis. - Implement transformer execution improvements, including token streaming, KV cache management, and batching. - Work with ML colleagues on quantization, sparsity, and mixed-precision execution; maintain unit, performance, acceptance, and regression tests in CI.

Requirements - At least 7 years of professional C++ development focused on performance-critical software. - Strong grasp of C++ templates and modern memory management. - Practical SIMD experience, such as AVX-512 or SSE, including intrinsics-based vectorization. - Experience with high-performance computing and parallel programming techniques.

Benefits and work setup Remote-first work with company-provided computer and home office setup. Benefits include company-paid medical, dental, vision, life, and disability coverage; supplemental coverage options; unlimited paid time off; 13 paid holidays; equity; and a 401(k) with matching. Applicants must already be authorized to work in the U.S.; new visa sponsorship is unavailable, though eligible H-1B transfers may be supported.