Skip to content
Test Linux

test_runner: support expecting a test-case to fail #79101

Sign in to view logs

test_runner: support expecting a test-case to fail

test_runner: support expecting a test-case to fail #79101

Triggered via pull request December 21, 2025 19:18
@JakobJingleheimerJakobJingleheimer
synchronize #60669
JakobJingleheimer:test_runner/feat/should-fail
Status Failure
Total duration 26m 56s
Artifacts –

test-linux.yml

on: pull_request
Matrix: test-linux
Fit to window
Zoom out
Zoom in

Annotations

4 errors and 2 notices
test-linux (ubuntu-24.04-arm)
Process completed with exit code 2.
test-linux (ubuntu-24.04-arm): test/test-runner/test-output-output.mjs#L0
--- stderr --- node:internal/modules/run_main:107 triggerUncaughtException( ^ AssertionError [ERR_ASSERTION]: Expected values to be strictly equal: + actual - expected + 'TAP version 13\n' + + '# Subtest: sync expect fail (method)\n' + + 'ok 1 - sync expect fail (method) # EXPECTED FAILURE\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: sync expect fail (options)\n' + + 'ok 2 - sync expect fail (options) # EXPECTED FAILURE\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: async expect fail (method)\n' + + 'ok 3 - async expect fail (method) # EXPECTED FAILURE\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: async expect fail (options)\n' + + 'ok 4 - async expect fail (options) # EXPECTED FAILURE\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: sync todo with expect fail\n' + + 'ok 5 - sync todo with expect fail # TODO\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: sync skip expect fail\n' + + 'ok 6 - sync skip expect fail # SKIP\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: sync pass todo\n' + + 'ok 7 - sync pass todo # TODO\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: sync pass todo with message\n' + + 'ok 8 - sync pass todo with message # TODO this is a passing todo\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: sync fail todo\n' + + 'not ok 9 - sync fail todo # TODO\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + " location: '/test/fixtures/test-runner/output/output.js:(LINE):1'\n" + + " failureType: 'testCodeFailure'\n" + + " error: 'thrown from sync fail todo'\n" + + " code: 'ERR_TEST_FAILURE'\n" + + ' stack: |-\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' ...\n' + + '# Subtest: sync fail todo with message\n' + + 'not ok 10 - sync fail todo with message # TODO this is a failing todo\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + " location: '/test/fixtures/test-runner/output/output.js:(LINE):1'\n" + + " failureType: 'testCodeFailure'\n" + + " error: 'thrown from sync fail todo with message'\n" + + " code: 'ERR_TEST_FAILURE'\n" + + ' stack: |-\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' ...\n' + + '# Subtest: sync skip pass\n' + + 'ok 11 - sync skip pass # SKIP\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: sync skip pass with message\n' + + 'ok 12 - sync skip pass with message # SKIP this is skipped\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: sync pass\n' + + 'ok 13 - sync pass\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# this test should pass\n' + + '# Subtest: sync throw fail\n' + + 'not ok 14 - sync throw fail\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + " location: '/test/fixtures/test-runner/output/output.js:(LINE):1'\n" + + " failureType: 'testCodeFailure'\n" + + " error: 'thrown from sync throw fail'\n" + + " code: 'ERR_TEST_FAILURE'\n" + + ' stack: |-\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' ...\n' + + '# Subtest: async skip pass\n' + + 'ok 15 - async skip pass # SKIP\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: async pass\n' + + 'ok 16 - async pass\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n
test-linux (ubuntu-24.04)
Process completed with exit code 2.
test-linux (ubuntu-24.04): test/test-runner/test-output-output.mjs#L0
--- stderr --- node:internal/modules/run_main:107 triggerUncaughtException( ^ AssertionError [ERR_ASSERTION]: Expected values to be strictly equal: + actual - expected + 'TAP version 13\n' + + '# Subtest: sync expect fail (method)\n' + + 'ok 1 - sync expect fail (method) # EXPECTED FAILURE\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: sync expect fail (options)\n' + + 'ok 2 - sync expect fail (options) # EXPECTED FAILURE\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: async expect fail (method)\n' + + 'ok 3 - async expect fail (method) # EXPECTED FAILURE\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: async expect fail (options)\n' + + 'ok 4 - async expect fail (options) # EXPECTED FAILURE\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: sync todo with expect fail\n' + + 'ok 5 - sync todo with expect fail # TODO\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: sync skip expect fail\n' + + 'ok 6 - sync skip expect fail # SKIP\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: sync pass todo\n' + + 'ok 7 - sync pass todo # TODO\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: sync pass todo with message\n' + + 'ok 8 - sync pass todo with message # TODO this is a passing todo\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: sync fail todo\n' + + 'not ok 9 - sync fail todo # TODO\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + " location: '/test/fixtures/test-runner/output/output.js:(LINE):1'\n" + + " failureType: 'testCodeFailure'\n" + + " error: 'thrown from sync fail todo'\n" + + " code: 'ERR_TEST_FAILURE'\n" + + ' stack: |-\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' ...\n' + + '# Subtest: sync fail todo with message\n' + + 'not ok 10 - sync fail todo with message # TODO this is a failing todo\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + " location: '/test/fixtures/test-runner/output/output.js:(LINE):1'\n" + + " failureType: 'testCodeFailure'\n" + + " error: 'thrown from sync fail todo with message'\n" + + " code: 'ERR_TEST_FAILURE'\n" + + ' stack: |-\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' ...\n' + + '# Subtest: sync skip pass\n' + + 'ok 11 - sync skip pass # SKIP\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: sync skip pass with message\n' + + 'ok 12 - sync skip pass with message # SKIP this is skipped\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: sync pass\n' + + 'ok 13 - sync pass\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# this test should pass\n' + + '# Subtest: sync throw fail\n' + + 'not ok 14 - sync throw fail\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + " location: '/test/fixtures/test-runner/output/output.js:(LINE):1'\n" + + " failureType: 'testCodeFailure'\n" + + " error: 'thrown from sync throw fail'\n" + + " code: 'ERR_TEST_FAILURE'\n" + + ' stack: |-\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' *\n' + + ' ...\n' + + '# Subtest: async skip pass\n' + + 'ok 15 - async skip pass # SKIP\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n" + + ' ...\n' + + '# Subtest: async pass\n' + + 'ok 16 - async pass\n' + + ' ---\n' + + ' duration_ms: *\n' + + " type: 'test'\n
sccache stats
0% - 0 hits, 0 misses, 0 errors
sccache stats
0% - 0 hits, 0 misses, 0 errors