(test) Fix unstable tests: LeaseServiceProviderTest, LeaseMaintainerFactoryTest, LeaseSystemTest, LeaseExpiryConnectionKillingThreadTest

[mathieucarbou]

LeaseServiceProviderTest

Junit test results: junit-test-results.zip

The problem is the assertion:

verify(timeSource, timeout(1000L)).sleep(200L);

Which expects that the LeaseExpiryConnectionKillingThread has been started and has called timeSource.sleep(200L); within 1 sec.

But we only got 1 interaction: timeSource.nanoTime(),which is called following LeaseResult leaseResult = service.acquireLease(clientDescriptor);

The test could be changed to instead include a latch in TestTimeSource and have the test wait for the expected call to happen, and remove the timeout.

LeaseMaintainerFactoryTest

Junit test results: junit-test-results.zip

Same issue : timeout() should not be used in verification because there is no guarantee that the thread will be started within this time

LeaseSystemTest

Junit test results: junit-test-results.zip

    LeaseTestUtil.waitForValidLease(leaseMaintainer);

    Lease lease1 = leaseMaintainer.getCurrentLease();
    Thread.sleep(1000L);
    Lease lease2 = leaseMaintainer.getCurrentLease();

    assertTrue(lease2.isValidAndContiguous(lease1));

The test runs slowly so the lease is not refreshed by the background thread (LeaseMaintenanceThread ). I think the assertion is wrong and we should ensure that the background thread refreshes correctly first.

LeaseExpiryConnectionKillingThreadTest

Junit test results:

• junit-test-results.zip
• junit-test-results (1).zip

Same timeout issue:

verify(timeSource, timeout(1000L).times(1)).sleep(200L);

There is no guarantee that the thread will start and call sleep(200) within 1 sec

[mathieucarbou]

Note from Chris:

we use the junit test retry rule in TC-EE for this kind of thing. You run it with really short leases to see if the machine can pass the test in that way
then we retry with longer lease timeouts if it fails