MOC-R4PCC02U12 (H100) has GPU and CPU thermal issues and won't power on

[jtriley]

Motivation

MOC-R4PCC02U12 ESI host in nerc-admins project is currently part of the nerc-ocp-prod cluster. During an intensive user-workload the host powered off and is showing SYS board issues related to GPU and CPU thermal events.

Completion criteria

Maintenance scheduled with Lenovo

References

See nerc-project/operations#1397 (comment) and additional comments after that for further details.

[hakasapl]

I have opened a ticket with Lenovo for this node:
Serial #: J70127BT
Ticket #: 3000704450

[joachimweyl]

@hakasapl would you please push Lenovo for an expected repair date? We have RH and NAIRR projects requesting more and more H100s. We want to get as many as we can back in action.

[hakasapl]

@joachimweyl this case already escalating on the Lenovo side. I am going to try a physical reseat today (already did virtual remotely).