Skip to content

Commit 1d923cf

Browse files
committed
Add contributor documentation for testing
1 parent 07aff16 commit 1d923cf

File tree

1 file changed

+35
-5
lines changed

1 file changed

+35
-5
lines changed

CONTRIBUTING.md

Lines changed: 35 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,49 @@
1-
# Development environment
1+
## Development environment
22

3-
## Nix
3+
### Nix
44

55
This repository has a nix flake that has everything you will need to run the tests.
66
You can just use `nix develop` or [`direnv`](https://direnv.net/) with the provided `.envrc`.
77
Notice that it uses a newer version of Scryer Prolog from the git repo than the one provided by
88
nixpkgs, so you may need to compile it, which may take some time. You can use the same binary
99
cache as the CI with `cachix use pkg-pl` to get the precompiled environment.
1010

11-
## Non-Nix
11+
### Non-Nix
1212

1313
You will need a [Scryer Prolog](https://github.com/mthom/scryer-prolog) built close to `master`
1414
(`0.9.4`, the stable release, is too old for what we are doing here),
1515
[just](https://github.com/casey/just) and git.
1616

17-
# Running the tests
17+
## Running the tests
18+
19+
You can run `just test` in the root directory and it will run all the tests, same as CI. You can
20+
also run `just ci` to run the entire CI.
21+
22+
## Writing tests
23+
24+
### Snapshot tests
25+
26+
The snapshot tests are in the `tests/snapshot/` directory. The idea is that we run an arbitrary
27+
script and check that everything it does what we expect.
28+
29+
A test case is composed of the following files in the `tests/snapshot/cases/` directory:
30+
31+
- The test script `test_name.sh` that runs the test.
32+
- (Optional) A `test_name.stdin` file whose contents will be passed to the test script in stdin.
33+
- (Optional) A `test_name.in` directory where the script will be run. Can be used to provide files
34+
for the test to operate on.
35+
- (Can be generated) A `test_name.stdout` with the expected stdout of the test.
36+
- (Can be generated) A `test_name.stderr` with the expected stderr of the test.
37+
- (Can be generated) A `test_name.status` with the expected exit status code of the test.
38+
39+
The last 3 files can be generated from the current behavior of the test with `just snapshot`.
40+
41+
In the `tests/snapshot/` directory, you can use the following just commands to deal with snapshot
42+
tests:
43+
44+
- `just test` actually runs and checks the tests.
45+
- `just dump` runs the tests and puts the generated files in a `dump/` directory for inspection
46+
without checking them. Useful for manual inspection.
47+
- `just snapshot` runs the tests and puts the generated files in the `cases/` directory.
48+
Useful if you are pretty sure the current behavior is correct.
1849

19-
You can run `just test` in the root directory and it will run all the tests, same as CI.

0 commit comments

Comments
 (0)