Commit 4ee6f9f
Reduce flakiness of space-specific ASG test
The current "binding a space-specific ASG" test fails approximately 80%
of the time on a vanilla Cloud Foundry deployment. In a nutshell, the
new space-specific ASG has not propagated by the time that the app has
restarted, and the app is beholden to the old ASGs: What should be a
timeout is a "connection refused".
This commit fixes that by inserting a 60-second delay between
setting the new ASG and restarting the app. The 60-second delay is not
arbitrary; rather, it is the result of the meticulous gathering of
empirical data presented in the chart below:
|Delay (seconds)|# of tests|Success Rate (%)|Success Rate Histogram |
|--------------:|---------:|---------------:|:--------------------------|
| 0 | 60 | 20 | ***** |
| 5 | 95 | 28 | ******* |
| 10 | 75 | 41 | ********** |
| 15 | 70 | 38 | ********* |
| 20 | 95 | 46 | *********** |
| 25 | 75 | 68 | ***************** |
| 30 | 120 | 63 | *************** |
| 35 | 60 | 76 | ******************* |
| 40 | 35 | 85 | ********************* |
| 45 | 20 | 85 | ********************* |
| 50 | 30 | 100 | ************************* |
| 55 | 35 | 100 | ************************* |
| 60 | 20 | 100 | ************************* |
| 65 | 30 | 100 | ************************* |
| 70 | 40 | 100 | ************************* |
Although 50 seconds should have been enough, we added another ten
seconds for no good reason other than headroom.
Fixes:
```
[Fail] [tasks] v3 tasks when associating a task with an app and binding a space-specific ASG [It] applies the associated app's ASGs to the task
/Users/cunnie/workspace/cf-acceptance-tests/tasks/task.go:355
```
```
2022-05-30T10:26:44.86-0700 [APP/TASK/woof/0] ERR 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to 10.0.244.255 port 80: Connection refused
```1 parent 1ccdda4 commit 4ee6f9f
1 file changed
+2
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
328 | 328 | | |
329 | 329 | | |
330 | 330 | | |
| 331 | + | |
| 332 | + | |
331 | 333 | | |
332 | 334 | | |
333 | 335 | | |
| |||
0 commit comments