Skip to content

Commit 33b7678

Browse files
9999yearsMikolaj
authored andcommitted
Update cabal-testsuite README.md
1 parent 2d7b84d commit 33b7678

File tree

1 file changed

+36
-17
lines changed

1 file changed

+36
-17
lines changed

cabal-testsuite/README.md

Lines changed: 36 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,21 @@
11
cabal-testsuite is a suite of integration tests for Cabal-based
22
frameworks.
33

4-
How to run
5-
----------
4+
# How to run
65

76
1. Build `cabal-testsuite` (`cabal build cabal-testsuite:cabal-tests`)
87
2. Run the `cabal-tests` executable. It will scan for all tests
98
in your current directory and subdirectories and run them.
10-
To run a specific set of tests, use `cabal-tests --with-cabal=CABALBIN PATH ...`.
11-
(e.g. `cabal run cabal-testsuite:cabal-tests -- --with-cabal=cabal cabal-testsuite/PackageTests/TestOptions/setup.test.hs`)
12-
You can control parallelism using the `-j` flag.
139

1410
There are a few useful flags:
1511

12+
* To run a specific set of tests, pass the path to a `.test.hs` file to run or
13+
use the `-p`/`--pattern` flag to filter tests.
14+
15+
See the ["Selecting tests"](#selecting-tests) section below for more details.
16+
17+
* `-j INT` controls the number of threads used for running tests.
18+
1619
* `--with-cabal PATH` can be used to specify the path of a
1720
`cabal-install` executable. IF YOU DO NOT SPECIFY THIS FLAG,
1821
CABAL INSTALL TESTS WILL NOT RUN.
@@ -28,6 +31,28 @@ There are a few useful flags:
2831
* `--keep-tmp-files` can be used to keep the temporary directories that tests
2932
are run in.
3033

34+
## Selecting tests
35+
36+
To run a specific set of tests, use `cabal-tests --with-cabal=CABALBIN PATH ...`, e.g.:
37+
38+
```
39+
cabal run cabal-testsuite:cabal-tests -- \
40+
--with-cabal=cabal \
41+
cabal-testsuite/PackageTests/TestOptions/setup.test.hs
42+
```
43+
44+
Alternatively, use `-p`/`--pattern` to select tests dynamically:
45+
46+
```
47+
cabal run cabal-testsuite:cabal-tests -- \
48+
--with-cabal=cabal \
49+
--pattern "/TestOptions/"
50+
```
51+
52+
See [the documentation for Tasty pattern
53+
syntax](https://hackage.haskell.org/package/tasty#patterns) for more
54+
information.
55+
3156
## Which Cabal library version do cabal-install tests use?
3257

3358
By default the `cabal-install` tests will use the `Cabal` library which comes with
@@ -74,8 +99,7 @@ components have broken doctests
7499
our CI currently checks that `Cabal-syntax` and `Cabal` doctests pass via
75100
`make doctest-install && make doctest` (you can use this `make`-based workflow too).
76101

77-
How to write
78-
------------
102+
# How to write
79103

80104
If you learn better by example, just look at the tests that live
81105
in `cabal-testsuite/PackageTests`; if you `git log -p`, you can
@@ -155,8 +179,7 @@ allow multiple tests to be defined in one file but run in parallel;
155179
at the moment, these just indicate long running tests that should
156180
be run early (to avoid straggling).
157181

158-
Frequently asked questions
159-
--------------------------
182+
# Frequently asked questions
160183

161184
For all of these answers, to see examples of the functions in
162185
question, grep the test suite.
@@ -223,8 +246,7 @@ long before editing a file, in order for file system timestamp
223246
resolution to pick it up. Use `withDelay` and `delay` prior to
224247
making a modification.
225248

226-
Hermetic tests
227-
--------------
249+
# Hermetic tests
228250

229251
Tests are run in a fresh temporary system directory. This attempts to isolate the
230252
tests from anything specific to do with your directory structure. In particular
@@ -235,8 +257,7 @@ tests from anything specific to do with your directory structure. In particular
235257
* You must `git add` all files which are relevant to the test, otherwise
236258
they will not be copied.
237259

238-
Design notes
239-
------------
260+
# Design notes
240261

241262
This is the second rewrite of the integration testing framework. The
242263
primary goal was to use Haskell as the test language (letting us take
@@ -296,8 +317,7 @@ figure out how to get out the threading setting, and then spawn
296317
that many GHCi servers to service the running threads. Improvements
297318
welcome.
298319

299-
Expect tests
300-
------------
320+
# Expect tests
301321

302322
An expect test (aka _golden test_)
303323
is a test where we read out the output of the test
@@ -366,8 +386,7 @@ Some other notes:
366386
on the output for the string you're looking for. Try to be
367387
deterministic, but sometimes it's not (easily) possible.
368388

369-
Non-goals
370-
---------
389+
# Non-goals
371390

372391
Here are some things we do not currently plan on supporting:
373392

0 commit comments

Comments
 (0)