Skip to content

Commit 4ee6f9f

Browse files
cunniectlong
authored andcommitted
Reduce flakiness of space-specific ASG test
The current "binding a space-specific ASG" test fails approximately 80% of the time on a vanilla Cloud Foundry deployment. In a nutshell, the new space-specific ASG has not propagated by the time that the app has restarted, and the app is beholden to the old ASGs: What should be a timeout is a "connection refused". This commit fixes that by inserting a 60-second delay between setting the new ASG and restarting the app. The 60-second delay is not arbitrary; rather, it is the result of the meticulous gathering of empirical data presented in the chart below: |Delay (seconds)|# of tests|Success Rate (%)|Success Rate Histogram | |--------------:|---------:|---------------:|:--------------------------| | 0 | 60 | 20 | ***** | | 5 | 95 | 28 | ******* | | 10 | 75 | 41 | ********** | | 15 | 70 | 38 | ********* | | 20 | 95 | 46 | *********** | | 25 | 75 | 68 | ***************** | | 30 | 120 | 63 | *************** | | 35 | 60 | 76 | ******************* | | 40 | 35 | 85 | ********************* | | 45 | 20 | 85 | ********************* | | 50 | 30 | 100 | ************************* | | 55 | 35 | 100 | ************************* | | 60 | 20 | 100 | ************************* | | 65 | 30 | 100 | ************************* | | 70 | 40 | 100 | ************************* | Although 50 seconds should have been enough, we added another ten seconds for no good reason other than headroom. Fixes: ``` [Fail] [tasks] v3 tasks when associating a task with an app and binding a space-specific ASG [It] applies the associated app's ASGs to the task /Users/cunnie/workspace/cf-acceptance-tests/tasks/task.go:355 ``` ``` 2022-05-30T10:26:44.86-0700 [APP/TASK/woof/0] ERR 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to 10.0.244.255 port 80: Connection refused ```
1 parent 1ccdda4 commit 4ee6f9f

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

tasks/task.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -328,6 +328,8 @@ exit 1`
328328
workflowhelpers.AsUser(TestSetup.AdminUserContext(), Config.DefaultTimeoutDuration(), func() {
329329
Expect(cf.Cf("bind-security-group", securityGroupName, TestSetup.RegularUserContext().Org, "--space", TestSetup.RegularUserContext().Space).Wait()).To(Exit(0))
330330
})
331+
// Give time to allow the security group to propagate
332+
time.Sleep(60 * time.Second)
331333

332334
By("restarting the app to apply the ASG")
333335
Expect(cf.Cf("restart", appName).Wait(Config.CfPushTimeoutDuration())).To(Exit(0))

0 commit comments

Comments
 (0)