AI Inference Kernel Optimization Engineer

Role overview Improve the speed and serving cost of open-model inference by optimizing GPU kernels, memory use, and production runtimes. You’ll work across low-level performance engineering and deployed serving systems, with results measured in latency and token cost.

Responsibilities - Build and tune kernels for operations such as attention, matrix multiplication, and normalization. - Improve data movement through cache reuse, KV-cache management, and operator fusion. - Implement low-precision inference paths, including INT4, FP8, and INT8, for the available hardware. - Profile GPU execution, inspect PTX or SASS where needed, and ship optimizations based on findings. - Integrate improvements into inference runtimes and contribute appropriate changes upstream. - Coordinate with research and platform engineers to support new model architectures.

Requirements - At least five years in systems software or ML systems engineering, including hands-on GPU performance work. - Strong C++ and Python skills, with deep experience in at least one of CUDA, Triton, or CUTLASS. - Ability to use hardware-level profiling tools and interpret low-level GPU execution. - Familiarity with a modern inference runtime such as vLLM or TensorRT-LLM, and with transformer inference performance and memory behavior. - Demonstrated kernel optimization work supported by before-and-after performance measurements.