Added a buffer period in the node performance tests

The node-kubelet-flaky e2e job that runs the the
`Node Performance Testing [Serial] [Slow] [Flaky]` e2e tests have been
flaking because of inconsistencies on the cpu manager checkpoint file.
This seems to be caused because the checkpoint file is deleted (which is
what needs to happen in order to change the CPU manager policy which is
used for these e2e tests) right after the e2e tests asserts that a pod
does not exist anymore.
However, after a pod is deleted, the CPU manager may still be cleaning
up the resources used by the pod which may result in the checkpoint file
being created.
Whenever this happened, the kubelet would panic if we then try to
subsequently change the CPU manager policy to "static" from "none" or
vice versa (this is done 4 times in these tests).

Signed-off-by: alejandrox1 <[email protected]>

Author: alejandrox1

test/e2e_node/node_perf_test.go

@@ -81,6 +81,15 @@ var _ = SIGDescribe("Node Performance Testing [Serial] [Slow] [Flaky]", func() {
 			GracePeriodSeconds: &gp,
 		}
 		f.PodClient().DeleteSync(pod.Name, delOpts, framework.DefaultPodDeletionTimeout)
+
+		// We are going to give some more time for the CPU manager to do any clean
+		// up it needs to do now that the pod has been deleted. Otherwise we may
+		// run into a data race condition in which the PostTestExec function
+		// deletes the CPU manager's checkpoint file while the CPU manager is still
+		// doing work and we end with a new checkpoint file after PosttestExec has
+		// finished. This issues would result in the kubelet panicking after we try
+		// and set the kubelet config.
+		time.Sleep(15 * time.Second)
 		ginkgo.By("running the post test exec from the workload")
 		err := wl.PostTestExec()
 		framework.ExpectNoError(err)