1
1
cabal-testsuite is a suite of integration tests for Cabal-based
2
2
frameworks.
3
3
4
- How to run
5
- ----------
4
+ # How to run
6
5
7
6
1 . Build ` cabal-testsuite ` (` cabal build cabal-testsuite:cabal-tests ` )
8
7
2 . Run the ` cabal-tests ` executable. It will scan for all tests
9
8
in your current directory and subdirectories and run them.
10
- To run a specific set of tests, use ` cabal-tests --with-cabal=CABALBIN PATH ... ` .
11
- (e.g. ` cabal run cabal-testsuite:cabal-tests -- --with-cabal=cabal cabal-testsuite/PackageTests/TestOptions/setup.test.hs ` )
12
- You can control parallelism using the ` -j ` flag.
13
9
14
10
There are a few useful flags:
15
11
12
+ * To run a specific set of tests, pass the path to a ` .test.hs ` file to run or
13
+ use the ` -p ` /` --pattern ` flag to filter tests.
14
+
15
+ See the [ "Selecting tests"] ( #selecting-tests ) section below for more details.
16
+
17
+ * ` -j INT ` controls the number of threads used for running tests.
18
+
16
19
* ` --with-cabal PATH ` can be used to specify the path of a
17
20
` cabal-install ` executable. IF YOU DO NOT SPECIFY THIS FLAG,
18
21
CABAL INSTALL TESTS WILL NOT RUN.
@@ -28,6 +31,28 @@ There are a few useful flags:
28
31
* ` --keep-tmp-files ` can be used to keep the temporary directories that tests
29
32
are run in.
30
33
34
+ ## Selecting tests
35
+
36
+ To run a specific set of tests, use ` cabal-tests --with-cabal=CABALBIN PATH ... ` , e.g.:
37
+
38
+ ```
39
+ cabal run cabal-testsuite:cabal-tests -- \
40
+ --with-cabal=cabal \
41
+ cabal-testsuite/PackageTests/TestOptions/setup.test.hs
42
+ ```
43
+
44
+ Alternatively, use ` -p ` /` --pattern ` to select tests dynamically:
45
+
46
+ ```
47
+ cabal run cabal-testsuite:cabal-tests -- \
48
+ --with-cabal=cabal \
49
+ --pattern "/TestOptions/"
50
+ ```
51
+
52
+ See [ the documentation for Tasty pattern
53
+ syntax] ( https://hackage.haskell.org/package/tasty#patterns ) for more
54
+ information.
55
+
31
56
## Which Cabal library version do cabal-install tests use?
32
57
33
58
By default the ` cabal-install ` tests will use the ` Cabal ` library which comes with
@@ -74,8 +99,7 @@ components have broken doctests
74
99
our CI currently checks that ` Cabal-syntax ` and ` Cabal ` doctests pass via
75
100
` make doctest-install && make doctest ` (you can use this ` make ` -based workflow too).
76
101
77
- How to write
78
- ------------
102
+ # How to write
79
103
80
104
If you learn better by example, just look at the tests that live
81
105
in ` cabal-testsuite/PackageTests ` ; if you ` git log -p ` , you can
@@ -155,8 +179,7 @@ allow multiple tests to be defined in one file but run in parallel;
155
179
at the moment, these just indicate long running tests that should
156
180
be run early (to avoid straggling).
157
181
158
- Frequently asked questions
159
- --------------------------
182
+ # Frequently asked questions
160
183
161
184
For all of these answers, to see examples of the functions in
162
185
question, grep the test suite.
@@ -223,8 +246,7 @@ long before editing a file, in order for file system timestamp
223
246
resolution to pick it up. Use ` withDelay ` and ` delay ` prior to
224
247
making a modification.
225
248
226
- Hermetic tests
227
- --------------
249
+ # Hermetic tests
228
250
229
251
Tests are run in a fresh temporary system directory. This attempts to isolate the
230
252
tests from anything specific to do with your directory structure. In particular
@@ -235,8 +257,7 @@ tests from anything specific to do with your directory structure. In particular
235
257
* You must ` git add ` all files which are relevant to the test, otherwise
236
258
they will not be copied.
237
259
238
- Design notes
239
- ------------
260
+ # Design notes
240
261
241
262
This is the second rewrite of the integration testing framework. The
242
263
primary goal was to use Haskell as the test language (letting us take
@@ -296,8 +317,7 @@ figure out how to get out the threading setting, and then spawn
296
317
that many GHCi servers to service the running threads. Improvements
297
318
welcome.
298
319
299
- Expect tests
300
- ------------
320
+ # Expect tests
301
321
302
322
An expect test (aka _ golden test_ )
303
323
is a test where we read out the output of the test
@@ -366,8 +386,7 @@ Some other notes:
366
386
on the output for the string you're looking for. Try to be
367
387
deterministic, but sometimes it's not (easily) possible.
368
388
369
- Non-goals
370
- ---------
389
+ # Non-goals
371
390
372
391
Here are some things we do not currently plan on supporting:
373
392
0 commit comments