Customer Reliability Engineer

Role overview Help customers keep large AI training and inference workloads running reliably on GPU clusters. You will investigate incidents across Linux and Kubernetes, coordinate with infrastructure providers to resolve issues, and build monitoring and automation that catch recurring problems earlier. The role is customer-facing and includes participation in a 24/7 on-call rotation.

Responsibilities - Own customer issues from reproduction and diagnosis through resolution or escalation, keeping users informed along the way. - Troubleshoot Linux processes, networking, storage, kernel logs, resource contention, and system services. - Diagnose Kubernetes scheduling, pod, node, and resource problems, including workloads that fail across multiple nodes. - Investigate GPU-related failures such as driver or device-plugin issues, hardware error reports, and thermal throttling; take operational steps such as cordoning or draining nodes when needed. - Work with external infrastructure providers to establish the cause of incidents and drive fixes. - Improve monitoring and write scripts that automate repetitive work or surface problems before customers report them.

Requirements - Practical comfort working in a Linux shell and understanding Kubernetes operations. - Ability to write automation in Python or Bash. - Clear written communication, including explaining technical problems calmly to customers under stress. - Sound incident judgment: prioritize investigation, know when to escalate, and provide updates while diagnosis is underway. - Willingness to participate in a 24/7 on-call rotation.

Nice to have - Production experience with NVIDIA GPUs, drivers, CUDA, DCGM, or Kubernetes device plugins. - Familiarity with InfiniBand, RoCE, NCCL, HPC workloads, or schedulers such as Slurm. - Customer-facing technical experience in support engineering, technical account management, solutions, or professional services. - Monitoring or infrastructure automation experience with tools such as Prometheus, Grafana, Datadog, Terraform, Ansible, or Helm. - Interest in AI infrastructure and large-scale training workloads.

Benefits and work setup - Remote role covering EMEA; full-time. - Compensation with equity, healthcare, dental and vision coverage for employees and dependents, 401(k), and unlimited paid time off.