Skip to content

test: skip tests that don't exactly match status code due to platform differences #3076

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 26, 2025
Merged

test: skip tests that don't exactly match status code due to platform differences #3076

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1618ec9
test: skip tests that don't exactly match status code due to platform…
pieh Aug 26, 2025
a37f10c
Merge branch 'main' into test-skip-not-matching-status-code
pieh Aug 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions tests/test-config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -396,6 +396,22 @@
"tests": [
"router autoscrolling on navigation bugs Should apply scroll when loading.js is used"
]
},
{
"file": "test/e2e/app-dir/actions-unrecognized/actions-unrecognized.test.ts",
"reason": "Mismatch of reported status code of response (500 instead of 'expected' 405). Test itself has FIXME comment about that.",
"tests": [
"unrecognized server actions should error and log a warning when submitting a server action with an unrecognized ID - nodejs server action invoked via form - js disabled",
"unrecognized server actions should error and log a warning when submitting a server action with an unrecognized ID - edge server action invoked via form - js disabled"
]
},
Comment on lines +400 to +407
Copy link
Contributor Author

@pieh pieh Aug 26, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See https://github.com/vercel/next.js/blob/canary/test/e2e/app-dir/actions-unrecognized/actions-unrecognized.test.ts#L154-L157 (it's same test running in 2 variants)

FAIL default test/e2e/app-dir/actions-unrecognized/actions-unrecognized.test.ts (93.755 s)
  unrecognized server actions
    should error and log a warning when submitting a server action with an unrecognized ID - nodejs
      ✓ server action invoked via form - js enabled (1555 ms)
      ✕ server action invoked via form - js disabled (569 ms)
      ✓ server action invoked from JS - with simple argument (632 ms)
      ✓ server action invoked from JS - with complex argument (354 ms)
    should error and log a warning when submitting a server action with an unrecognized ID - edge
      ✓ server action invoked via form - js enabled (1000 ms)
      ✕ server action invoked via form - js disabled (405 ms)
      ✓ server action invoked from JS - with simple argument (641 ms)
      ✓ server action invoked from JS - with complex argument (438 ms)

  ● unrecognized server actions › should error and log a warning when submitting a server action with an unrecognized ID - nodejs › server action invoked via form - js disabled

    expect(received).toBe(expected) // Object.is equality

    Expected: 405
    Received: 500

    �[0m �[90m 154 |�[39m             �[90m// FIXME: When deployed to vercel, the request is logged as a 500, but returns a 405.�[39m
     �[90m 155 |�[39m             �[90m// We also don't seem to display the error page correctly�[39m
    �[31m�[1m>�[22m�[39m�[90m 156 |�[39m             expect(response�[33m.�[39mstatus())�[33m.�[39mtoBe(�[35m405�[39m)
     �[90m     |�[39m                                       �[31m�[1m^�[22m�[39m
     �[90m 157 |�[39m             expect(response�[33m.�[39mheaders()[�[32m'content-type'�[39m])�[33m.�[39mtoStartWith(�[32m'text/html'�[39m)
     �[90m 158 |�[39m           } �[36melse�[39m {
     �[90m 159 |�[39m             �[90m// FIXME: Currently, an unrecognized id in an MPA action results in a 500.�[39m�[0m

      at toBe (e2e/app-dir/actions-unrecognized/actions-unrecognized.test.ts:156:39)
      at e2e/app-dir/actions-unrecognized/actions-unrecognized.test.ts:205:11

  ● unrecognized server actions › should error and log a warning when submitting a server action with an unrecognized ID - edge › server action invoked via form - js disabled

    expect(received).toBe(expected) // Object.is equality

    Expected: 405
    Received: 500

    �[0m �[90m 154 |�[39m             �[90m// FIXME: When deployed to vercel, the request is logged as a 500, but returns a 405.�[39m
     �[90m 155 |�[39m             �[90m// We also don't seem to display the error page correctly�[39m
    �[31m�[1m>�[22m�[39m�[90m 156 |�[39m             expect(response�[33m.�[39mstatus())�[33m.�[39mtoBe(�[35m405�[39m)
     �[90m     |�[39m                                       �[31m�[1m^�[22m�[39m
     �[90m 157 |�[39m             expect(response�[33m.�[39mheaders()[�[32m'content-type'�[39m])�[33m.�[39mtoStartWith(�[32m'text/html'�[39m)
     �[90m 158 |�[39m           } �[36melse�[39m {
     �[90m 159 |�[39m             �[90m// FIXME: Currently, an unrecognized id in an MPA action results in a 500.�[39m�[0m

      at toBe (e2e/app-dir/actions-unrecognized/actions-unrecognized.test.ts:156:39)
      at e2e/app-dir/actions-unrecognized/actions-unrecognized.test.ts:205:11

{
"file": "test/e2e/app-dir/actions/app-action-size-limit-invalid.test.ts",
"reason": "Mismatch of reported status code of response (502 instead of expected 500)",
"tests": [
"app-dir action size limit invalid config should respect the size set in serverActions.bodySizeLimit for plaintext fetch actions should error for requests that exceed the size limit",
"app-dir action size limit invalid config should respect the size set in serverActions.bodySizeLimit for multipart fetch actions should error for requests that exceed the size limit"
]
Comment on lines +408 to +414
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

FAIL default test/e2e/app-dir/actions/app-action-size-limit-invalid.test.ts (136.783 s)
  app-dir action size limit invalid config
    should respect the size set in serverActions.bodySizeLimit for plaintext fetch actions
      ✓ should not error for requests that stay below the size limit (6499 ms)
      ✕ should error for requests that exceed the size limit (978 ms)
    should respect the size set in serverActions.bodySizeLimit for multipart fetch actions
      ✓ should not error for requests that stay below the size limit (5812 ms)
      ✓ should not error for requests that are at the size limit (3270 ms)
      ✕ should error for requests that exceed the size limit (1315 ms)

  ● app-dir action size limit invalid config › should respect the size set in serverActions.bodySizeLimit for plaintext fetch actions › should error for requests that exceed the size limit

    expect(received).toBe(expected) // Object.is equality

    Expected: 500
    Received: 502

    �[0m �[90m 137 |�[39m         { request�[33m:�[39m { method�[33m:�[39m �[32m'POST'�[39m�[33m,�[39m pathname�[33m:�[39m �[32m'/file'�[39m } }
     �[90m 138 |�[39m       )
    �[31m�[1m>�[22m�[39m�[90m 139 |�[39m       expect(actionResponse�[33m.�[39mstatus())�[33m.�[39mtoBe(�[35m500�[39m) �[90m// TODO: 413?�[39m
     �[90m     |�[39m                                       �[31m�[1m^�[22m�[39m
     �[90m 140 |�[39m       expect(
     �[90m 141 |�[39m         �[36mawait�[39m actionResponse�[33m.�[39mrequest()�[33m.�[39mheaderValue(�[32m'content-type'�[39m)
     �[90m 142 |�[39m       )�[33m.�[39mtoStartWith(�[32m'text/plain'�[39m)�[0m

      at Object.toBe (e2e/app-dir/actions/app-action-size-limit-invalid.test.ts:139:39)

  ● app-dir action size limit invalid config › should respect the size set in serverActions.bodySizeLimit for multipart fetch actions › should error for requests that exceed the size limit

    expect(received).toBe(expected) // Object.is equality

    Expected: 500
    Received: 502

    �[0m �[90m 226 |�[39m         { request�[33m:�[39m { method�[33m:�[39m �[32m'POST'�[39m�[33m,�[39m pathname�[33m:�[39m �[32m'/form'�[39m } }
     �[90m 227 |�[39m       )
    �[31m�[1m>�[22m�[39m�[90m 228 |�[39m       expect(actionResponse�[33m.�[39mstatus())�[33m.�[39mtoBe(�[35m500�[39m) �[90m// TODO: 413?�[39m
     �[90m     |�[39m                                       �[31m�[1m^�[22m�[39m
     �[90m 229 |�[39m       expect(
     �[90m 230 |�[39m         �[36mawait�[39m actionResponse�[33m.�[39mrequest()�[33m.�[39mheaderValue(�[32m'content-type'�[39m)
     �[90m 231 |�[39m       )�[33m.�[39mtoStartWith(�[32m'multipart/form-data'�[39m)�[0m

      at Object.toBe (e2e/app-dir/actions/app-action-size-limit-invalid.test.ts:228:39)

}
],
"failures": [
Expand Down
Loading