-
Notifications
You must be signed in to change notification settings - Fork 1.1k
docs: Adds documentation for the 112
exit code when posix-compliant exit codes are enabled.
#6294
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
cacieprins
wants to merge
7
commits into
release/15.5.0
Choose a base branch
from
docs/exit-112-network-err
base: release/15.5.0
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+27
−70
Open
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
76b9478
docs: exit code 112 for cloud api network errors
cacieprins 572f35d
docs: detailed exit code scenarios
cacieprins ac3451d
changelog
cacieprins 00ea77e
fix ref for modulus behavior on test fail count
cacieprins 2b0bd11
edit pass
cacieprins aa15022
Update changelog.mdx
cacieprins b760279
Update changelog.mdx
cacieprins File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -442,14 +442,12 @@ cypress run --port 8080 | |
|
||
#### `cypress run --posix-exit-codes` {#cypress-run-posix-exit-codes} | ||
|
||
This changes the default behavior of Cypress regarding exit codes. This flag causes Cypress to exit with POSIX exit code of 1 when there are test failures or if Cypress could not run, rather than the number of tests that failed. | ||
This changes the default behavior of Cypress regarding exit codes. This flag causes Cypress to exit with POSIX exit code of 1 when there are test failures or if Cypress could not run, rather than the number of tests that failed. It can return additional POSIX-compliant exit codes for other scenarios as well. See the [Exit Code section](/app/references/command-line#Exit-code) for a comprehensive reference of Cypress exit codes. | ||
|
||
```shell | ||
cypress run --posix-exit-codes | ||
``` | ||
|
||
For more about Cypress' exit code behavior, see the [Exit Code section](/app/references/command-line#Exit-code). | ||
|
||
#### `cypress run --project <project-path>` {#cypress-run-project-lt-project-path-gt} | ||
|
||
To see this in action we've set up an | ||
|
@@ -598,69 +596,20 @@ Cypress Cloud will display any tags sent with the appropriate run. | |
|
||
#### Exit code | ||
|
||
When Cypress finishes running tests, it exits. If there are no failed tests, the | ||
exit code will be 0. | ||
|
||
```text | ||
## All tests pass | ||
$ cypress run | ||
... | ||
Tests Passing Failing | ||
✔ All specs passed! 00:16 17 17 0 | ||
|
||
## print exit code on Mac or Linux | ||
$ echo $? | ||
0 | ||
``` | ||
|
||
By default, if there are any test failures, then the exit code will match the number of | ||
tests that failed. | ||
|
||
```text | ||
## Spec with two failing tests | ||
$ cypress run | ||
... | ||
Tests Passing Failing | ||
✖ 1 of 1 failed (100%) 00:22 17 14 2 | ||
|
||
## print exit code on Mac or Linux | ||
$ echo $? | ||
2 | ||
``` | ||
Cypress supports two different exit code behaviors. The default behavior is similar to that of Mocha, but is not POSIX compliant: reserved exit codes can be returned for certain conditions. As well, the default behavior | ||
can return a false negative when there are a number of tests that fail in multiples of 256: most terminals will apply modulo 256 to any exit code larger than 255. | ||
|
||
You can change this behavior by passing the `--posix-exit-codes` flag. When this flag is passed, | ||
the exit code will be 0 if all tests pass. If there are any test failures or if Cypress could not run, | ||
the exit code will be 1. | ||
POSIX compliant exit codes are available via the `--posix-exit-codes` flag. When this flag is passed, additional conditions can be returned that are not possible with the default behavior, including when Cypress fails due to | ||
network errors when connections to Cypress Cloud are required, such as in `--record` or `--parallel` runs. | ||
|
||
```text | ||
## Spec with two failing tests | ||
$ cypress run | ||
... | ||
Tests Passing Failing | ||
✖ 1 of 1 failed (100%) 00:22 17 14 2 | ||
When using POSIX compliant exit codes, the number of failed tests can be determined from [structured reporter data](/app/tooling/reporters). | ||
|
||
## print exit code on Mac or Linux | ||
$ echo $? | ||
1 | ||
``` | ||
|
||
If Cypress could not run for some reason (for example if no spec files were | ||
found) then the exit code will be 1. | ||
|
||
```text | ||
## No spec files found | ||
$ cypress run --spec not-found.js | ||
... | ||
Can't run because no spec files were found. | ||
|
||
We searched for any files matching this glob pattern: | ||
|
||
not-found.js | ||
|
||
## print exit code on Mac or Linux | ||
$ echo $? | ||
1 | ||
``` | ||
| Exit Condition | POSIX Exit Code (with `--posix-exit-codes`) | Default Exit Code (without `--posix-exit-codes`) | | ||
| ------------------------------------------------------------------------------------------- | ------------------------------------------- | ------------------------------------------------ | | ||
| All tests pass | 0 | 0 | | ||
| _n_ tests fail | 1 | _n_ | | ||
| Cypress could not run due to a network error when connections to Cypress Cloud are required | 112 | 1 | | ||
| Cypress could not run for any other reason, including no spec files found | 1 | 1 | | ||
|
||
### `cypress open` | ||
|
||
|
@@ -1000,10 +949,11 @@ and with Windows CMD and PowerShell terminal windows. | |
|
||
## History | ||
|
||
| Version | Changes | | ||
| ------------------------------------------ | ---------------------------------------------------------- | | ||
| [15.4.0](/app/references/changelog#15-4-0) | Added `--posix-exit-codes` flag to `cypress run` | | ||
| [12.6.0](/app/references/changelog#12-6-0) | Added `--auto-cancel-after-failures` flag to `cypress run` | | ||
| [5.4.0](/app/references/changelog) | Added `prune` subcommand to `cypress cache` | | ||
| [5.4.0](/app/references/changelog) | Added `--size` flag to `cypress cache list` subcommand | | ||
| [4.9.0](/app/references/changelog) | Added `--quiet` flag to `cypress run` | | ||
| Version | Changes | | ||
| ------------------------------------------ | -------------------------------------------------------------------------------------------------------- | | ||
| [15.5.0](/app/references/changelog#15-5-0) | Modified exit code documentations from narrative to a reference table, adding the 112 exit code scenario | | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Generally the history table is used to document the changes to Cypress, rather than the changes to the documentation. So here the change would be something like |
||
| [15.4.0](/app/references/changelog#15-4-0) | Added `--posix-exit-codes` flag to `cypress run` | | ||
| [12.6.0](/app/references/changelog#12-6-0) | Added `--auto-cancel-after-failures` flag to `cypress run` | | ||
| [5.4.0](/app/references/changelog) | Added `prune` subcommand to `cypress cache` | | ||
| [5.4.0](/app/references/changelog) | Added `--size` flag to `cypress cache list` subcommand | | ||
| [4.9.0](/app/references/changelog) | Added `--quiet` flag to `cypress run` | |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The text should probably come out of PR cypress-io/cypress#32635 otherwise it is unclear where the definitive text for the release notes is supposed to be reviewed.