Skip to content

Commit 2dc6a10

Browse files
kubafloCopilot
andcommitted
Fix false PASSED on XHarness app crash (exit 83)
When XHarness crashes with exit code 83 (APP_LAUNCH_FAILURE), the test runner outputs 'Passed: 0 / Failed: 0'. The parser was not detecting this as an env error because: 1. The regex 'APP_LAUNCH_FAILURE|exit code:? 83' didn't match the actual format 'XHarness exit code: 83 (APP_LAUNCH_FAILURE)' 2. 'Passed: 0 / Failed: 0' fell through all checks to the generic 'Could not parse' path, but in some flows was treated as PASSED Fixes: - Add 'XHarness exit code: 83' as explicit env error pattern - Add 'Application test run crashed' as env error pattern - Guard: 'Passed: 0 + Failed: 0' = env error (zero tests ran) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 8a393cb commit 2dc6a10

File tree

1 file changed

+7
-0
lines changed

1 file changed

+7
-0
lines changed

.github/skills/verify-tests-fail-without-fix/scripts/verify-tests-fail.ps1

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -555,6 +555,8 @@ function Get-TestResultFromOutput {
555555
$envErrorPatterns = @(
556556
@{ Pattern = "error ADB0010.*InstallFailedException"; Message = "App install failed (ADB broken pipe)" }
557557
@{ Pattern = "APP_LAUNCH_FAILURE|exit code:?\s*83"; Message = "App failed to launch (XHarness exit 83)" }
558+
@{ Pattern = "XHarness exit code:\s*83"; Message = "App failed to launch (XHarness exit 83)" }
559+
@{ Pattern = "Application test run crashed"; Message = "App crashed during test run" }
558560
@{ Pattern = "SIGABRT.*load_aot_module"; Message = "App crashed during AOT loading" }
559561
@{ Pattern = "AppiumServerHasNotBeenStartedLocally"; Message = "Appium server failed to start" }
560562
@{ Pattern = "no such element.*could not be located"; Message = "Test element not found (app may not have loaded)" }
@@ -680,6 +682,11 @@ function Get-TestResultFromOutput {
680682
}
681683
}
682684

685+
# Zero tests ran (Passed: 0, Failed: 0) — treat as env error, not success
686+
if ($content -match "Passed:\s*0" -and $content -match "Failed:\s*0") {
687+
return @{ Passed = $false; EnvError = $true; Error = "Zero tests ran (Passed: 0, Failed: 0)"; Total = 0; Failed = 0; Skipped = 0 }
688+
}
689+
683690
return @{ Passed = $false; Error = "Could not parse test results"; Total = 0; Failed = 0; Skipped = 0 }
684691
}
685692

0 commit comments

Comments
 (0)