Remote job
Principal AI Network Engineer - Highrise AI
Job details
About this role
Role overview
Bring high-speed networks for large AI clusters from physical installation to validated production readiness. You’ll lead fabric commissioning, performance qualification, and technical acceptance, then provide senior escalation support and drive reliability improvements for production networks.
Responsibilities
- Commission cluster networks by initializing switches, validating configurations, and confirming host connectivity and production readiness. - Integrate Ethernet or InfiniBand compute fabrics, including RDMA/RoCE, with storage, management, and out-of-band networks. - Verify topology, routing, redundancy, addressing, port mappings, optics, cabling, and interoperability across switches and network cards. - Automate repeatable network deployments and maintain configuration and firmware baselines. - Test link health, bandwidth, latency, congestion, resilience, and scale; investigate workload results to identify communication bottlenecks. - Resolve production incidents through deep troubleshooting and root cause analysis, and support upgrades, expansion, failover exercises, and vendor escalations.
Requirements
- Strong hands-on expertise in Ethernet or InfiniBand fabrics, RDMA/RoCE, routing and switching, and data center network architectures. - Experience with high-speed networking, including 200/400/800G systems, network cards, optical links, telemetry, and performance testing. - Linux networking knowledge and experience automating network configuration with Python, Ansible, or comparable tools. - Ability to troubleshoot across cabling, optics, switches, network cards, transport, and workload performance. - Strong ownership of acceptance criteria, reliability, documentation, and follow-through; able to make technical decisions and collaborate across engineering teams. - Clear communication with engineering, site operations, customers, and vendors.
Nice to have
- Familiarity with NVIDIA networking and fabric management tools.
Benefits and work setup
Full-time engineering work across multiple infrastructure sites. The role is hybrid or remote depending on location, with required data center travel for commissioning and troubleshooting. Flexibility for critical incidents and maintenance outside standard hours is expected.