Remote job
AI Infrastructure Operations Engineer - Compute & Storage - Highrise AI
Job details
About this role
Role overview
Provide Tier 2 operational support for production GPU compute and storage platforms used in large-scale AI environments. The engineer owns escalated tickets from initial investigation through resolution or handoff, coordinating with operations, site technicians, and specialist teams to restore service and keep clusters running reliably.
Responsibilities
- Triage escalated incidents and service requests, assess severity and customer impact, and coordinate restoration within operational targets. - Diagnose compute and storage issues using logs, monitoring dashboards, management interfaces, and approved tools; record findings and provide actionable escalations. - Troubleshoot Linux servers, GPU alerts, boot and hardware faults, firmware or driver issues, and resource availability. - Support node provisioning, reimaging, configuration changes, maintenance, and return-to-service checks using established procedures. - Investigate Kubernetes or Slurm node and workload failures, and perform initial analysis of connectivity or GPU communication symptoms. - Monitor and troubleshoot distributed and high-performance storage, including capacity, throughput, availability, mounts, disk alerts, and client access; maintain accurate records and improve runbooks.
Requirements
- Practical experience troubleshooting enterprise compute hardware, operating systems, management interfaces such as BMC tools, and monitoring systems. - Working knowledge of storage technologies, such as NFS, distributed file systems, or object storage, and basic storage performance concepts. - Experience with ticketing and change management processes, plus Bash or Python scripting for diagnostics or routine automation. - Structured, evidence-based troubleshooting and judgment about when to resolve, collaborate, or escalate. - Strong ticket ownership, attention to operational detail, customer service, and written and verbal English communication. - Ability to work across NOC, reliability, hardware, network, storage, and engineering teams.
Nice to have
- Exposure to NVIDIA GPU systems, InfiniBand or RDMA, Kubernetes, Slurm, or AI/HPC storage platforms.
Benefits and work setup
This is a full-time role supporting infrastructure across multiple sites. The position may be hybrid or remote depending on location. Data-center travel for commissioning, acceptance, and complex troubleshooting is required, and flexibility for critical incidents or maintenance outside regular hours is expected. The work includes collaboration with operations and engineering specialists on upgrades, maintenance, and cluster expansion.