kola/multipath: wait for systemd firstboot target #4349
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request aims to fix a race condition in multipath tests by waiting for systemd's first-boot-complete.target before rebooting. The approach is sound, but the implementation of the waiting logic has a bug that would prevent it from working as intended. I've provided a detailed comment with a suggested fix for the waiting function. Once that is addressed, the change should effectively resolve the flakiness issue.
jbtrystram
force-pushed
the
mpath_wait_firstboot_complete
branch
2 times, most recently
from
87b1989 to
6c14eb0
Compare
| verifyMultipath(c, m, "/var/lib/containers") | ||
| } | ||
|
|
||
| func waitForCompleteFirstboot(c cluster.TestCluster) { |
There was a problem hiding this comment.
optional:
rather than running a retry here you could possibly run systemd-run --wait and then run that inside a util.WaitUntilReady like is done here.
i.e. rather than worrying about how many times to retry you can just focus on how much time you want to spend waiting.
There was a problem hiding this comment.
Thank for the suggestion. Applied it.
jbtrystram
force-pushed
the
mpath_wait_firstboot_complete
branch
2 times, most recently
from
c12a98d to
02bcc86
Compare
Although `mpath-var-lib-containers.service` is set to only run on first boot, it sometime runs twice when the system reboots too early. Sometimes, in low load CI environement, the reboot in this test happens before systemd's `first-boot-complete.target` is reached. This make `ConditionFirstBoot` to still be true at the next boot, causing the mpath service to fail, because it already ran during the actual first boot. A previous attempt[1] at fixing this improved the flake but this happened again and i noticed that systemd didn't reach this target before the reboot: `Reached target first-boot-complete.target - First Boot Complete` is only shown after the second boot in the logs. Likely fixes coreos/rhel-coreos-config#66 [1] coreos@abd0c18
jbtrystram
force-pushed
the
mpath_wait_firstboot_complete
branch
from
02bcc86 to
50e5cd6
Compare
2 participants
Although
mpath-var-lib-containers.serviceis set to only run on first boot, it sometime runs twice when the system reboots too early. Sometimes, in low load CI environement, the reboot in this test happens before systemd'sfirst-boot-complete.targetis reached. This makeConditionFirstBootto still be true at the next boot, causing the mpath service to fail, because it already ran during the actual first boot.A previous attempt[1] at fixing this improved the flake but this happened again and i noticed that systemd didn't reach this target before the reboot:
Reached target first-boot-complete.target - First Boot Completeis only shown after the second boot in the logs.Likely fixes coreos/rhel-coreos-config#66
[1] abd0c18