Commit c6a13a8
authored
test(webview): skip flaky test on min vscode #5955
## Problem
It appears that older versions of VSCode are causing this test to be
flaky, which appears to be fixed in v1.88. See findings section for more
information.
## Solution
- Skip the test if vscode version is pre-1.88.
- Add tech-debt test to remove this check once min is 1.88+.
## Findings
- Ran 1000 times in CI, and failed 29 times. (2.9% failure rate). If we
re-reran 3 times, ci would pass ~ 99.998% of the time (assuming failures
independent).
- Only fails on VSCode minimum (1.83.0).
- Originally thought it was related to activating Q extension or
fetching it from CDN, but appears mocking this still results in
flakiness.
- Every failure comes with a `ERROR:gles2_command_buffer_stub.cc(364)]
ContextResult::kFatalFailure Failed to create context.` which appears to
be from OpenGL. This means it is coming from very far down the stack
since VSCode is built on electron, which uses chromium, which is then
using OpenGL. (ai suggests this is a GPU problem?).
- Thought to be from race condition with async constructor hack, fixed
by #5961.
- Evidence is suggesting that this could be a bug in a prior version of
VSCode, electron, or OpenGL, and should go away once we bump minimum
VSCode version.
- Works on VSCode 1.90+, 1.87- fails, 1.88 is the minimum version it is
not flaky. (verified using 1000 attempts).
- Failures are NOT independent since retrying 10x times only reduced
failure rate to ~1%. Therefore retries are not a viable option.1 parent 10b3f1c commit c6a13a8
File tree
1 file changed
+6
-5
lines changed- packages/core/src/test/applicationcomposer/messageHandlers
1 file changed
+6
-5
lines changedLines changed: 6 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
| 11 | + | |
11 | 12 | | |
12 | 13 | | |
13 | | - | |
14 | | - | |
15 | | - | |
16 | | - | |
17 | | - | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| |||
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
| 35 | + | |
35 | 36 | | |
36 | 37 | | |
0 commit comments