System Software Engineer - AI

Role overview Develop system software that helps large AI models run efficiently across distributed GPU clusters. This role connects AI frameworks with low-level infrastructure by designing communication and execution primitives, investigating performance across hardware and software, and shaping foundational technology for distributed training and inference.

Responsibilities - Work across the software stack to shape infrastructure around the needs of emerging AI models. - Analyze distributed training and inference workloads to find performance bottlenecks. - Investigate issues at the software and hardware boundary. - Benchmark and characterize performance on multi-node clusters. - Help build and refine communication and execution primitives for large-scale workloads.

Requirements - At least five years of relevant experience. - Strong C++ and Python skills, with knowledge of memory management, concurrency, and operating system internals. - Proficiency developing in Linux environments. - Familiarity with networking protocols and concepts. - Bachelor's or master's degree in computer engineering, computer science, or a related field.

Nice to have - CUDA programming or performance optimization for parallel architectures. - Experience with distributed AI frameworks such as PyTorch, JAX, or DeepSpeed, or inference engines such as vLLM, SGLang, or TRT-LLM. - Hands-on experience with cluster orchestration and telemetry at scale. - Experience deploying production systems and managing releases.

Benefits and work setup - Hybrid work based in Palo Alto, California, or Charlottesville, Virginia. - Advertised target base salary of $140,000–$220,000 per year, plus equity, benefits, and a 401(k). The range depends on role level, experience, and location.