Skip to content

Commit b93143a

Browse files
martin-schulze-viresomartin-schulze-vireso
authored
Merge pull request bats-core#72 from mh182/feat/stderr
feat: add assert_stderr, refute_stderr, assert_stderr_line
2 parents 0ec504e + 03ca0ba commit b93143a

13 files changed

+1816
-81
lines changed

‎README.md‎

Lines changed: 242 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,8 @@ This project provides the following functions:
3939
- [assert_output](#assert_output) / [refute_output](#refute_output) Assert output does (or does not) contain given content.
4040
- [assert_line](#assert_line) / [refute_line](#refute_line) Assert a specific line of output does (or does not) contain given content.
4141
- [assert_regex](#assert_regex) / [refute_regex](#refute_regex) Assert a parameter does (or does not) match given pattern.
42+
- [assert_stderr](#assert_stderr) / [refute_stderr](#refute_stderr) Assert stderr does (or does not) contain given content.
43+
- [assert_stderr_line](#assert_stderr_line) / [refute_stderr_line](#refute_stderr_line) Assert a specific line of stderr does (or does not) contain given content.
4244

4345
These commands are described in more detail below.
4446

@@ -759,7 +761,7 @@ Fail if the value (first parameter) matches the pattern (second parameter).
759761

760762
On failure, the value, the pattern and the match are displayed.
761763

762-
```
764+
```bash
763765
@test 'refute_regex()' {
764766
refute_regex 'WhatsApp' 'What.'
765767
}
@@ -786,6 +788,245 @@ For description of the matching behavior, refer to the documentation of the
786788
> Thus, it's good practice to avoid using `BASH_REMATCH` in conjunction with `refute_regex()`.
787789
> The valuable information the array contains is the matching part of the value which is printed in the failing test log, as mentioned above._
788790
791+
### `assert_stderr`
792+
793+
> _**Note**:
794+
> `run` has to be called with `--separate-stderr` to separate stdout and stderr into `$output` and `$stderr`.
795+
> If not, `$stderr` will be empty, causing `assert_stderr` to always fail.
796+
797+
Similarly to `assert_output`, this function verifies that a command or function produces the expected stderr.
798+
The stderr matching can be literal (the default), partial or by regular expression.
799+
The expected stderr can be specified either by positional argument or read from STDIN by passing the `-`/`--stdin` flag.
800+
801+
#### Literal matching
802+
803+
By default, literal matching is performed.
804+
The assertion fails if `$stderr` does not equal the expected stderr.
805+
806+
```bash
807+
echo_err() {
808+
echo "$@" >&2
809+
}
810+
811+
@test 'assert_stderr()' {
812+
run --separate-stderr echo_err 'have'
813+
assert_stderr 'want'
814+
}
815+
816+
@test 'assert_stderr() with pipe' {
817+
run --separate-stderr echo_err 'hello'
818+
echo_err 'hello' | assert_stderr -
819+
}
820+
821+
@test 'assert_stderr() with herestring' {
822+
run --separate-stderr echo_err 'hello'
823+
assert_stderr - <<< hello
824+
}
825+
```
826+
827+
On failure, the expected and actual stderr are displayed.
828+
829+
```
830+
-- stderr differs --
831+
expected : want
832+
actual : have
833+
--
834+
```
835+
836+
#### Existence
837+
838+
To assert that any stderr exists at all, omit the `expected` argument.
839+
840+
```bash
841+
@test 'assert_stderr()' {
842+
run --separate-stderr echo_err 'have'
843+
assert_stderr
844+
}
845+
```
846+
847+
On failure, an error message is displayed.
848+
849+
```
850+
-- no stderr --
851+
expected non-empty stderr, but stderr was empty
852+
--
853+
```
854+
855+
#### Partial matching
856+
857+
Partial matching can be enabled with the `--partial` option (`-p` for short).
858+
When used, the assertion fails if the expected _substring_ is not found in `$stderr`.
859+
860+
```bash
861+
@test 'assert_stderr() partial matching' {
862+
run --separate-stderr echo_err 'ERROR: no such file or directory'
863+
assert_stderr --partial 'SUCCESS'
864+
}
865+
```
866+
867+
On failure, the substring and the stderr are displayed.
868+
869+
```
870+
-- stderr does not contain substring --
871+
substring : SUCCESS
872+
stderr : ERROR: no such file or directory
873+
--
874+
```
875+
876+
#### Regular expression matching
877+
878+
Regular expression matching can be enabled with the `--regexp` option (`-e` for short).
879+
When used, the assertion fails if the *extended regular expression* does not match `$stderr`.
880+
881+
*Note: The anchors `^` and `$` bind to the beginning and the end (respectively) of the entire stderr; not individual lines.*
882+
883+
```bash
884+
@test 'assert_stderr() regular expression matching' {
885+
run --separate-stderr echo_err 'Foobar 0.1.0'
886+
assert_stderr --regexp '^Foobar v[0-9]+\.[0-9]+\.[0-9]$'
887+
}
888+
```
889+
890+
On failure, the regular expression and the stderr are displayed.
891+
892+
```
893+
-- regular expression does not match stderr --
894+
regexp : ^Foobar v[0-9]+\.[0-9]+\.[0-9]$
895+
stderr : Foobar 0.1.0
896+
--
897+
```
898+
899+
### `refute_stderr`
900+
901+
> _**Note**:
902+
> `run` has to be called with `--separate-stderr` to separate stdout and stderr into `$output` and `$stderr`.
903+
> If not, `$stderr` will be empty, causing `refute_stderr` to always pass.
904+
905+
Similar to `refute_output`, this function verifies that a command or function does not produce the unexpected stderr.
906+
(It is the logical complement of `assert_stderr`.)
907+
The stderr matching can be literal (the default), partial or by regular expression.
908+
The unexpected stderr can be specified either by positional argument or read from STDIN by passing the `-`/`--stdin` flag.
909+
910+
### `assert_stderr_line`
911+
912+
> _**Note**:
913+
> `run` has to be called with `--separate-stderr` to separate stdout and stderr into `$output` and `$stderr`.
914+
> If not, `$stderr` will be empty, causing `assert_stderr_line` to always fail.
915+
916+
Similarly to `assert_stderr`, this function verifies that a command or function produces the expected stderr.
917+
It checks that the expected line appears in the stderr (default) or at a specific line number.
918+
Matching can be literal (default), partial or regular expression.
919+
This function is the logical complement of `refute_stderr_line`.
920+
921+
#### Looking for a line in the stderr
922+
923+
By default, the entire stderr is searched for the expected line.
924+
The assertion fails if the expected line is not found in `${stderr_lines[@]}`.
925+
926+
```bash
927+
echo_err() {
928+
echo "$@" >&2
929+
}
930+
931+
@test 'assert_stderr_line() looking for line' {
932+
run --separate-stderr echo_err $'have-0\nhave-1\nhave-2'
933+
assert_stderr_line 'want'
934+
}
935+
```
936+
937+
On failure, the expected line and the stderr are displayed.
938+
939+
```
940+
-- stderr does not contain line --
941+
line : want
942+
stderr (3 lines):
943+
have-0
944+
have-1
945+
have-2
946+
--
947+
```
948+
949+
#### Matching a specific line
950+
951+
When the `--index <idx>` option is used (`-n <idx>` for short), the expected line is matched only against the line identified by the given index.
952+
The assertion fails if the expected line does not equal `${stderr_lines[<idx>]}`.
953+
954+
```bash
955+
@test 'assert_stderr_line() specific line' {
956+
run --separate-stderr echo_err $'have-0\nhave-1\nhave-2'
957+
assert_stderr_line --index 1 'want-1'
958+
}
959+
```
960+
961+
On failure, the index and the compared stderr_lines are displayed.
962+
963+
```
964+
-- line differs --
965+
index : 1
966+
expected : want-1
967+
actual : have-1
968+
--
969+
```
970+
971+
#### Partial matching
972+
973+
Partial matching can be enabled with the `--partial` option (`-p` for short).
974+
When used, a match fails if the expected *substring* is not found in the matched line.
975+
976+
```bash
977+
@test 'assert_stderr_line() partial matching' {
978+
run --separate-stderr echo_err $'have 1\nhave 2\nhave 3'
979+
assert_stderr_line --partial 'want'
980+
}
981+
```
982+
983+
On failure, the same details are displayed as for literal matching, except that the substring replaces the expected line.
984+
985+
```
986+
-- no stderr line contains substring --
987+
substring : want
988+
stderr (3 lines):
989+
have 1
990+
have 2
991+
have 3
992+
--
993+
```
994+
995+
#### Regular expression matching
996+
997+
Regular expression matching can be enabled with the `--regexp` option (`-e` for short).
998+
When used, a match fails if the *extended regular expression* does not match the line being tested.
999+
1000+
*Note: As expected, the anchors `^` and `$` bind to the beginning and the end (respectively) of the matched line.*
1001+
1002+
```bash
1003+
@test 'assert_stderr_line() regular expression matching' {
1004+
run --separate-stderr echo_err $'have-0\nhave-1\nhave-2'
1005+
assert_stderr_line --index 1 --regexp '^want-[0-9]$'
1006+
}
1007+
```
1008+
1009+
On failure, the same details are displayed as for literal matching, except that the regular expression replaces the expected line.
1010+
1011+
```
1012+
-- regular expression does not match line --
1013+
index : 1
1014+
regexp : ^want-[0-9]$
1015+
line : have-1
1016+
--
1017+
```
1018+
1019+
### `refute_stderr_line`
1020+
1021+
> _**Note**:
1022+
> `run` has to be called with `--separate-stderr` to separate stdout and stderr into `$output` and `$stderr`.
1023+
> If not, `$stderr` will be empty, causing `refute_stderr_line` to always pass.
1024+
1025+
Similarly to `refute_stderr`, this function helps to verify that a command or function produces the correct stderr.
1026+
It checks that the unexpected line does not appear in the stderr (default) or in a specific line of it.
1027+
Matching can be literal (default), partial or regular expression.
1028+
This function is the logical complement of `assert_stderr_line`.
1029+
7891030
<!-- REFERENCES -->
7901031
7911032
[bats]: https://github.com/bats-core/bats-core

0 commit comments

Comments
 (0)