Skip to content

upgrade pre-checks do not report correct switch name #1226

@afewell-hh

Description

@afewell-hh

During an upgrade from 25.03 to 25.04, check of upgrade constraints found a problem with a switch in the fabric, but it reported the name of the problem switch as "agent leaf-01" rather than the name of the problem switch. I found a switch with a problem, but this leaves me feeling uneasy as, what if there is more than 1 problem, I dont know for sure which switch it found a problem is. It could be I found a different problem entirely.

There bad switch appears to have a hardware failure so will likely need RMA, in the meantime would like to upgrade the other switches by using a force flag on the upgrade - a separate feature request that would be nice is, a soft way to bring the problem switch offline as, if I use the force flag, I bypass other checks on other devices, there could be other problems.

Here is output that shows the switch and agent names and the incorrect name in the pre-check output:

core@control-1 ~/upgrade-25_04/control--control-1--install $ sudo ./hhfab-recipe upgrade
Dec 11 00:37:03.216 INF Hedgehog Fabricator Recipe version=v0.41.3
Dec 11 00:37:03.217 INF Running control node upgrade
Dec 11 00:37:08.320 ERR upgrading: running control upgrade: checking upgrade constraints: agent leaf-01 is not ready
core@control-1 ~/upgrade-25_04/control--control-1--install $ kubectl get switch
NAME                     PROFILE            ROLE          DESCR               GROUPS   AGE
800g-backend-leaf-u29    celestica-ds5000   server-leaf   Backend Leaf u29             147d
800g-backend-leaf-u32    celestica-ds5000   server-leaf   Backend Leaf u32             147d
800g-backend-leaf-u35    celestica-ds5000   server-leaf   Backend Leaf u35             147d
800g-backend-leaf-u38    celestica-ds5000   server-leaf   Backend Leaf u38             147d
800g-backend-spine-u41   celestica-ds5000   spine         Backend Spine u41            147d
800g-backend-spine-u44   celestica-ds5000   spine         Backend Spine u44            147d
core@control-1 ~/upgrade-25_04/control--control-1--install $ kubectl get agent
NAME                     ROLE          DESCR               APPLIED   APPLIEDG   CURRENTG   VERSION
800g-backend-leaf-u29    server-leaf   Backend Leaf u29    19m       22         22         v0.81.1
800g-backend-leaf-u32    server-leaf   Backend Leaf u32    43d       20         20         v0.81.1
800g-backend-leaf-u35    server-leaf   Backend Leaf u35    45m       17         17         v0.81.1
800g-backend-leaf-u38    server-leaf   Backend Leaf u38    13m       20         20         v0.81.1
800g-backend-spine-u41   spine         Backend Spine u41   11m       9          9          v0.81.1
800g-backend-spine-u44   spine         Backend Spine u44   13m       9          9          v0.81.1
leaf-01                  server-leaf   Backend Leaf 01                          13         
leaf-02                  server-leaf   Backend Leaf 02                          14         
spine-01                 spine         Backend Spine 01                         3          
spine-02                 spine         Backend Spine 02

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions