Remote job
Senior Systems Software Engineer - Data Center Performance
Job details
About this role
Role overview
Assess how new computing platforms perform under demanding HPC and AI workloads, from individual nodes through racks and full clusters. You will combine hands-on experiments, profiling, and software engineering to find bottlenecks, verify fixes, and help teams deliver reliable performance at scale.
Responsibilities
- Benchmark nodes, racks, and clusters with microbenchmarks and representative applications; measure throughput, latency, and scaling. - Diagnose Linux, compute, memory, storage, and network issues, including scheduling, NUMA placement, resource contention, and communication overhead. - Correlate application profiles, system traces, hardware counters, and cluster telemetry to identify root causes. - Build focused reproductions and work with hardware, firmware, driver, library, and application teams to implement and validate improvements. - Develop Python tools for benchmark automation, data collection, and analysis; maintain repeatable baselines and detect regressions. - Validate new platforms and deployments, investigate underperforming nodes or links, and confirm that corrective actions restore workload performance.
Requirements
- Extensive hands-on Linux performance analysis, including processes and threads, CPU and memory architecture, NUMA, I/O, and networking. - Experience investigating multi-node HPC or GPU-accelerated systems and distinguishing local bottlenecks from communication, imbalance, and shared-resource contention. - Deep expertise in at least one area such as GPU application performance, Linux systems, high-speed networking, parallel storage, or HPC performance tools. - Proficiency in Python and experience developing or debugging performance-sensitive C, C++, and CUDA code. - Ability to design controlled experiments, interpret profiling and counter data, and demonstrate reproducible performance gains. - Clear technical communication and experience leading work across hardware and software teams; a relevant degree or equivalent experience and 8+ years of applicable experience.
Nice to have
- Experience with Nsight tools, Linux perf, eBPF, RDMA, MPI, or NCCL, and interconnects such as PCIe, NVLink, InfiniBand, or Ethernet. - Familiarity with Slurm, containers, cgroups, and the effects of workload placement and CPU, GPU, and NIC affinity. - Experience optimizing distributed training, inference, or scientific workloads, or building automated performance regression systems.