Test flake in Flow Control EnqueueAndWait

[elevran]

What happened:
While testing change in unrelated code flow control test failed with error.
Rerunning the test passed.

--- FAIL: TestFlowController_EnqueueAndWait (0.01s)
    --- FAIL: TestFlowController_EnqueueAndWait/Lifecycle (0.00s)
        --- FAIL: TestFlowController_EnqueueAndWait/Lifecycle/OnReqCtxTimeoutAfterDistribution (0.05s)
            controller_test.go:853: 
                	Error Trace:	/home/prow/go/src/sigs.k8s.io/gateway-api-inference-extension/pkg/epp/flowcontrol/controller/controller_test.go:853
                	Error:      	"3.349078ms" is not greater than or equal to "20ms"
                	Test:       	TestFlowController_EnqueueAndWait/Lifecycle/OnReqCtxTimeoutAfterDistribution
                	Messages:   	EnqueueAndWait returned faster than the TTL allows, indicating the timer did not function correctly
FAIL

What you expected to happen:
Consistent pass/fail depending on PR.

How to reproduce it (as minimally and precisely as possible):
See this run on PR #1857

Anything else we need to know?:

Environment:

• Kubernetes version (use kubectl version):
• Inference extension version (use git describe --tags --dirty --always):
• Cloud provider or hardware configuration:
• Install tools:
• Others:

[nirrozenbaum]

cc: @LukeAVanDrie

[LukeAVanDrie]

/assign