@askervin I took a closer look at this review comment and the observed failure to allocate exclusive CPUs. Filed a fix for it as #604.

@klihub, I rebased this (few commits on top of the v0.11 tag). I verified that the failure is from runc as expected. The test passes with fixed runc.

Do you think it would be ok and not too ugly, if in this situation a test could state its own verdict (like SKIP) instead of letting the framework to decide the verdict based on script exit status (PASS=0, FAIL!=0) or test runner failure (ERROR: exit status could not be read due to test code runner exploding to pieces).

I know it would be better to introduce a "test_verdict_SKIP" or similar function to run.sh so that test scripts would not get creative with different verdicts. But I think that would be better to do in a separate PR that touches run_tests.sh, run.sh, this test code.var.sh, and possibly the CXL memory test that has been written to work only on fedora.

@klihub, I rebased this (few commits on top of the v0.11 tag). I verified that the failure is from runc as expected. The test passes with fixed runc.

Do you think it would be ok and not too ugly, if in this situation a test could state its own verdict (like SKIP) instead of letting the framework to decide the verdict based on script exit status (PASS=0, FAIL!=0) or test runner failure (ERROR: exit status could not be read due to test code runner exploding to pieces).

I know it would be better to introduce a "test_verdict_SKIP" or similar function to run.sh so that test scripts would not get creative with different verdicts. But I think that would be better to do in a separate PR that touches run_tests.sh, run.sh, this test code.var.sh, and possibly the CXL memory test that has been written to work only on fedora.

Yes, I think this should be fine. I don't think there is much difference, since anyway the tests need to get at least somewhat creative anyway to verify that some known external factor is causing test failure instead of the test itself failing.

I was thinking about doing the same but with checking what ended up in the OCI Spec instead cgroups, with something like this...

ctr=$(vm-command "crictl ps --name pod0c0 2>/dev/null" | grep Running | cut -d ' ' -f1)
cpus=$(vm-command "crictl inspect $ctr 2>/dev/null| jq .info.runtimeSpec.linux.resources.cpu.cpus" | sed -E 's/.([0-9,-]*)./\1/')
if (echo $cpus | tr ',' '\n' | grep -q '^4095$'); then
  error "Test verdict: SKIP, due to buggy runc..."
  exit 0
else
  error "failure..."

But if we can do an effectively similarly trustable verification in a simple way (what you seem to have in the PR now), then that's even better.