|
| 1 | +# Writing tests |
| 2 | + |
| 3 | +So you want to contribute a test? Great! Here's a quick summary. |
| 4 | + |
| 5 | +## Before you start |
| 6 | + |
| 7 | +Firstly, what language are you going to use to write your test? The |
| 8 | +principal languages in which tests are authored are C and Rust, which |
| 9 | +each have their own toolchain. These tests are in |
| 10 | +[`tests/c`](../tests/c/) and [`tests/rust`](../tests/rust/), |
| 11 | +respectively. We would be happy to add other WASI-targeting language |
| 12 | +toolchains. |
| 13 | + |
| 14 | +Secondly, what WASI version are you testing? Currently the WASI test |
| 15 | +suite has tests for WASI preview 1 (sometimes versioned as 0.1) and for |
| 16 | +WASI preview 3 (version 0.3.0, currently in pre-releases), corresponding |
| 17 | +to the `wasm32-wasip1` and `wasm32-wasip3` compiler targets. (Adding |
| 18 | +support for `wasm64` variants would be welcome.) Tests written in C |
| 19 | +typically target a standard POSIX API, leaving it up to the toolchain to |
| 20 | +select the WASI version used for the resulting binary; for this reason |
| 21 | +there is just one [`tests/c/src`](../tests/c/src) directory with all the |
| 22 | +tests, which may be compiled for different WASI targets. But different |
| 23 | +WASI versions have different APIs, and Rust tests are typically coded |
| 24 | +directly against them, with tests for the different versions in |
| 25 | +different directories, e.g. |
| 26 | +[`tests/rust/wasm32-wasip1`](../tests/rust/wasm32-wasip1) and |
| 27 | +[`tests/rust/wasm32-wasip3`](../tests/rust/wasm32-wasip3). |
| 28 | + |
| 29 | +## Make test source and (possibly) its JSON |
| 30 | + |
| 31 | +Next, what are you testing? Let's say you are testing some subtlety |
| 32 | +regarding `stat`, for WASIp3, in Rust. Well then you need to add your |
| 33 | +`filesystem-stat-subtlety.rs` to |
| 34 | +[`tests/rust/wasm32-wasip3/src/bin/`](../tests/rust/wasm32-wasip3/src/bin/), |
| 35 | +and it will be automatically built. (How? We'll come back to that.) |
| 36 | + |
| 37 | +What does your test need? If you are testing `stat`, probably you need |
| 38 | +a file, so the WASI module is going to need to start with a preopened |
| 39 | +directory. Make the directory, say call it `stat-working-dir`, as a |
| 40 | +subfolder of the directory your test is in. Then create a |
| 41 | +`filesystem-stat-subtlety.json` in the test directory: |
| 42 | + |
| 43 | +```json |
| 44 | +{ |
| 45 | + "dirs": ["stat-working-dir"] |
| 46 | +} |
| 47 | +``` |
| 48 | + |
| 49 | +A `dirs` declaration in a test's JSON file adds a dir to the preopens. |
| 50 | + |
| 51 | +If you need to create a file, name it something that ends with |
| 52 | +`.cleanup`, if possible, and ideally have the test case remove it at the |
| 53 | +end. But just in case, the test runner will delete files in a test's |
| 54 | +`dirs` with names like that, both before and after running the test. |
| 55 | + |
| 56 | + |
| 57 | +## Building tests |
| 58 | + |
| 59 | +Finally, we need to build the test. C tests are built by running |
| 60 | +[`tests/c/build.py`](../tests/c/build.py), and Rust tests via |
| 61 | +[`tests/rust/build.py`](../tests/rust/build.py). Those scripts will |
| 62 | +compile all the tests they have (all the <tt>tests/c/src/\*.c</tt> files |
| 63 | +and all the <tt>tests/rust/<i>target</i>/src/bin/\*.rs</tt> files), and |
| 64 | +copy the resulting wasm files to the generated build directory |
| 65 | +(<tt>tests/c/testsuite/<i>target</i>/</tt> or |
| 66 | +<tt>tests/rust/testsuite/<i>target</i>/</tt>). `build.py` will also |
| 67 | +copy directories if needed. Pass `--verbose --dry-run` to the |
| 68 | +`build.py` scripts to see what they are doing. |
| 69 | + |
| 70 | +Of course, you need a toolchain for those build scripts to work. |
| 71 | +Concretely, you need a toolchain from C or Rust that can target |
| 72 | +`wasm32-wasip1` or `wasm32-wasip3`. |
| 73 | + |
| 74 | +For C, one installs `wasi-sdk`. On x64-64 Ubuntu, that gives us: |
| 75 | + |
| 76 | +``` |
| 77 | +export WASI_SDK=wasi-sdk-27.0-x86_64-linux |
| 78 | +curl -LO https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-27/$WASI_SDK.tar.gz |
| 79 | +tar xvf $WASI_SDK.tar.gz |
| 80 | +export WASI_SDK_PATH=`pwd`/$WASI_SDK |
| 81 | +export CC="${WASI_SDK_PATH}/bin/clang" |
| 82 | +export CXX="${WASI_SDK_PATH}/bin/clang++" |
| 83 | +``` |
| 84 | + |
| 85 | +For Rust, one uses `cargo`. Use [rustup](https://rustup.rs/): |
| 86 | + |
| 87 | +``` |
| 88 | +rustup update |
| 89 | +rustup toolchain install stable |
| 90 | +rustup toolchain install nightly # needed for wasip3, for now |
| 91 | +rustup default stable |
| 92 | +rustup +stable target add wasm32-wasip1 |
| 93 | +rustup +nightly target add wasm32-wasip2 # for wasip3 |
| 94 | +``` |
| 95 | + |
| 96 | +Note that until wasip3 is released, we use the wasip2 toolchain for |
| 97 | +wasip3. |
| 98 | + |
| 99 | +Now you can run `build.py`. For Rust, run as `./build.py |
| 100 | +--toolchain=wasm32-wasip3:nightly`, so as to use the nightly channel |
| 101 | +instead of the default stable, for wasip3. |
| 102 | + |
| 103 | +## Final notes |
| 104 | + |
| 105 | +Sounds gnarly, and in a way it is: cross-compiling for multiple targets |
| 106 | +from multiple languages has some necessary complexity. When in doubt, |
| 107 | +do like other tests. When really in doubt, take a look to see what the |
| 108 | +[CI workflows](../.github/workflows/compile-tests.yml) do. Finally, see |
| 109 | +the [full test case specification](./specification.md) for full details |
| 110 | +on the JSON test metadata format. Good luck! |
0 commit comments