-
Notifications
You must be signed in to change notification settings - Fork 7
Description
During an upgrade from 25.03 to 25.04, check of upgrade constraints found a problem with a switch in the fabric, but it reported the name of the problem switch as "agent leaf-01" rather than the name of the problem switch. I found a switch with a problem, but this leaves me feeling uneasy as, what if there is more than 1 problem, I dont know for sure which switch it found a problem is. It could be I found a different problem entirely.
There bad switch appears to have a hardware failure so will likely need RMA, in the meantime would like to upgrade the other switches by using a force flag on the upgrade - a separate feature request that would be nice is, a soft way to bring the problem switch offline as, if I use the force flag, I bypass other checks on other devices, there could be other problems.
Here is output that shows the switch and agent names and the incorrect name in the pre-check output:
core@control-1 ~/upgrade-25_04/control--control-1--install $ sudo ./hhfab-recipe upgrade
Dec 11 00:37:03.216 INF Hedgehog Fabricator Recipe version=v0.41.3
Dec 11 00:37:03.217 INF Running control node upgrade
Dec 11 00:37:08.320 ERR upgrading: running control upgrade: checking upgrade constraints: agent leaf-01 is not ready
core@control-1 ~/upgrade-25_04/control--control-1--install $ kubectl get switch
NAME PROFILE ROLE DESCR GROUPS AGE
800g-backend-leaf-u29 celestica-ds5000 server-leaf Backend Leaf u29 147d
800g-backend-leaf-u32 celestica-ds5000 server-leaf Backend Leaf u32 147d
800g-backend-leaf-u35 celestica-ds5000 server-leaf Backend Leaf u35 147d
800g-backend-leaf-u38 celestica-ds5000 server-leaf Backend Leaf u38 147d
800g-backend-spine-u41 celestica-ds5000 spine Backend Spine u41 147d
800g-backend-spine-u44 celestica-ds5000 spine Backend Spine u44 147d
core@control-1 ~/upgrade-25_04/control--control-1--install $ kubectl get agent
NAME ROLE DESCR APPLIED APPLIEDG CURRENTG VERSION
800g-backend-leaf-u29 server-leaf Backend Leaf u29 19m 22 22 v0.81.1
800g-backend-leaf-u32 server-leaf Backend Leaf u32 43d 20 20 v0.81.1
800g-backend-leaf-u35 server-leaf Backend Leaf u35 45m 17 17 v0.81.1
800g-backend-leaf-u38 server-leaf Backend Leaf u38 13m 20 20 v0.81.1
800g-backend-spine-u41 spine Backend Spine u41 11m 9 9 v0.81.1
800g-backend-spine-u44 spine Backend Spine u44 13m 9 9 v0.81.1
leaf-01 server-leaf Backend Leaf 01 13
leaf-02 server-leaf Backend Leaf 02 14
spine-01 spine Backend Spine 01 3
spine-02 spine Backend Spine 02