Skip to content

Commit d15fda3

Browse files
alexcrichtonalexcrichton
authored
Spruce up the README of this repository (#1387)
* Spruce up the README of this repository The previous iteration had good information in it but felt a bit bland and lacking. I've added more information such as: * Examples * Updating some descriptions * Contributing documentation * Testing documentation * CI documentation * Fuzzing documentation * Split out a CONTRIBUTING.md
1 parent 60ff869 commit d15fda3

File tree

2 files changed

+283
-13
lines changed

2 files changed

+283
-13
lines changed

‎CONTRIBUTING.md

Lines changed: 177 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,177 @@
1+
# Contributing
2+
3+
This project welcomes contributions for bug fixes, documentation updates, new
4+
features, or whatever you might like. Development is done through GitHub pull
5+
requests. Feel free to reach out on the [Bytecode Alliance
6+
Zulip](https://bytecodealliance.zulipchat.com/) as well if you'd like assistance
7+
in contributing or would just like to say hi.
8+
9+
### Building From Source
10+
11+
To create a debug build of this project from source, execute this command at the
12+
root of the repository:
13+
14+
```
15+
$ cargo build
16+
```
17+
18+
And the resulting binary is located at `./target/debug/wasm-tools` for the
19+
current platform.
20+
21+
An optimized build can be produced with:
22+
23+
```
24+
$ cargo build --release
25+
```
26+
27+
### Testing
28+
29+
Many crates in this repository (located in `crates/*`) both have unit tests
30+
(`#[test]` functions throughout the source) and integration tests
31+
(`crates/*/tests/*.rs`). Testing an individual crate can be done with:
32+
33+
```
34+
$ cargo test -p wasmparser
35+
```
36+
37+
Running all tests can be done by fist ensuring that the spec test suite is
38+
checked out:
39+
40+
```
41+
$ git submodule update --init
42+
```
43+
44+
and then using Cargo to execute all tests:
45+
46+
```
47+
$ cargo test --workspace
48+
```
49+
50+
Running the spec test suite can be done with:
51+
52+
```
53+
$ cargo test --test roundtrip
54+
```
55+
56+
and running a single spec test can be done with an argument to this command as a
57+
string filter on the filename.
58+
59+
```
60+
$ cargo test --test roundtrip binary-leb128.wast
61+
```
62+
63+
Many tests are also located in the top-level `tests/*` folder. This is organized
64+
into a few suites:
65+
66+
* `tests/cli/*` - these files are run by the `tests/cli.rs` test file and are
67+
intended to be tests for the CLI itself. They start with `;; RUN: ...` headers
68+
to indicate what commands should run and adjacent files indicate the expected
69+
output.
70+
71+
* `tests/local/*` - these are handwritten `*.wat` and `*.wast` tests. The
72+
`*.wat` files must all validate as valid modules and `*.wast` files run their
73+
directives in the same manner as the spec test suite. This folder additional
74+
subfolders for specific classes of tests, for example `missing-features` has
75+
all optional wasm features disabled to test what happens when a feature is
76+
implemented but disabled at runtime. The `component-model` folder contains all
77+
tests related to enabling the component model feature.
78+
79+
* `tests/testsuite` - this is a git submodule pointing to the [upstream test
80+
suite repository](https://github.com/WebAssembly/testsuite/) and is where spec
81+
tests come from.
82+
83+
* `tests/roundtrip.rs` - this is the main driver for the `local` and `testsuite`
84+
folders. This will crawl over all files in those folders and execute what
85+
tests it can. This means running `*.wast` directives such as `assert_invalid`.
86+
Additionally all valid wasm modules are printed with `wasmprinter` and then
87+
parsed again with `wat` to ensure that they can be round-tripped through the
88+
crates.
89+
90+
* `tests/snapshots` - this contains golden output files which correspond to the
91+
`wasmprinter`-printed version of binaries of all tests. These files are used
92+
to view the impact of changes to `wasmprinter`.
93+
94+
Many tests throughout the repository have automatically generated files
95+
associated with them which reflect the expected output of an operation. This is
96+
done to view, during code review, the impact of changes made. It's not expected
97+
that these files need to be edited by hand, but instead setting the environment
98+
variable `BLESS=1` when running tests will update all of these test
99+
expectations.
100+
101+
### Continuous Integration
102+
103+
All changes to `wasm-tools` are required to pass the CI suite powered by GitHub
104+
Actions. Pull requests will automatically have checks performed and can only be
105+
merged once all tests are passing. CI checks currently include:
106+
107+
* Code is all formatted correctly (use `cargo fmt` locally to pass this)
108+
* Tests pass on Rust stable, beta, and Nightly.
109+
* Tests pass on Linux, macOS, and Windows.
110+
* This tool can be compiled to WebAssembly using the `wasm32-wasi` target.
111+
* Fuzzers can be built.
112+
* Various miscellaneous checks such as building the tool with various
113+
combinations of Cargo features.
114+
115+
### Fuzzing
116+
117+
This repository uses LLVM's libFuzzer through the [`cargo
118+
fuzz`](https://github.com/rust-fuzz/cargo-fuzz) tool. Building fuzzers requires
119+
a Nightly Rust toolchain which can be acquired with Rustup-based installations
120+
of Rust by executing:
121+
122+
```
123+
$ rustup update nightly
124+
```
125+
126+
Next the `cargo-fuzz` runner should be installed:
127+
128+
```
129+
$ cargo install cargo-fuzz
130+
```
131+
132+
Fuzzers are then built with:
133+
134+
```
135+
$ cargo +nightly fuzz build
136+
```
137+
138+
Useful options to this can include:
139+
140+
* `--dev` - build fuzzers in debug mode instead of release mode (default is
141+
release)
142+
* `--sanitizer none` - Rust doesn't benefit much from AddressSanitizer for
143+
example so disabling sanitizers can improve fuzzing performance and build more
144+
quickly too.
145+
146+
The fuzzing binary for this project is located at
147+
`target/$host_target/release/run`. Due to limitations on OSS-Fuzz all fuzzers
148+
are combined into a single binary at this time. This binary can be run with:
149+
150+
```
151+
$ cargo +nightly fuzz run run
152+
```
153+
154+
The main driver for fuzzing is located at `fuzz/fuzz_targets/run.rs`. This
155+
driver dispatches, based on the input, to a number of other fuzzers. Each
156+
individual fuzzer lives in `fuzz/src/*.rs`.
157+
158+
Running a single fuzzer can be done by configuring the `FUZZER` environment
159+
variable:
160+
161+
```
162+
$ FUZZER=roundtrip cargo +nightly fuzz run run
163+
```
164+
165+
More documentation of `cargo fuzz` can [be found
166+
online](https://rust-fuzz.github.io/book/cargo-fuzz.html).
167+
168+
# License
169+
170+
This project is licensed under the Apache 2.0 license with the LLVM exception.
171+
See [LICENSE](LICENSE) for more details.
172+
173+
### Contribution
174+
175+
Unless you explicitly state otherwise, any contribution intentionally submitted
176+
for inclusion in this project by you, as defined in the Apache-2.0 license,
177+
shall be licensed as above, without any additional terms or conditions.

‎README.md

Lines changed: 106 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -4,21 +4,23 @@
44
<strong>A <a href="https://bytecodealliance.org/">Bytecode Alliance</a> project</strong>
55

66
<p>
7-
<strong>Rust tooling for low-level manipulation of WebAssembly modules</strong>
7+
<strong>CLI and Rust libraries for low-level manipulation of WebAssembly modules</strong>
88
</p>
99
</div>
1010

1111
# Installation
1212

13-
This project can be installed and compiled from source with this Cargo command:
13+
[Precompiled artifacts built on CI][artifacts] are available for download for
14+
each release.
15+
16+
If you'd prefer to build from source then first [install Rust for your
17+
platform](https://www.rust-lang.org/tools/install) and then use the included
18+
Cargo package manager to install:
1419

1520
```
1621
$ cargo install wasm-tools
1722
```
1823

19-
Additionally there are [precompiled artifacts built on CI][artifacts] which are
20-
available for download as well.
21-
2224
[artifacts]: https://github.com/bytecodealliance/wasm-tools/releases
2325

2426
Installation can be confirmed with:
@@ -33,18 +35,99 @@ Subcommands can be explored with:
3335
$ wasm-tools help
3436
```
3537

38+
# Examples
39+
40+
Basic validation/printing:
41+
42+
```sh
43+
# Validate a WebAssembly file
44+
$ wasm-tools validate foo.wasm
45+
46+
# Validate a WebAssembly module in the text format, automatically converting to
47+
# binary.
48+
$ wasm-tools validate foo.wat
49+
50+
# Validate a WebAssembly file enabling an off-by-default feature
51+
$ wasm-tools validate foo.wasm --features=exception-handling
52+
53+
# Validate a WebAssembly file with a default-enabled feature disabled
54+
$ wasm-tools validate foo.wasm --features=-simd
55+
56+
# Print the text format of a module to stdout
57+
$ wasm-tools print foo.wasm
58+
59+
# Convert a binary module to text
60+
$ wasm-tools print foo.wasm -o foo.wat
61+
```
62+
63+
Simple mutation as well as piping commands together:
64+
65+
```sh
66+
# Mutate a WebAssembly module and print its text representation to stdout
67+
$ wasm-tools mutate foo.wasm -t
68+
69+
# Mutate a WebAssembly module with a non-default seed and validate that the
70+
# output is a valid module.
71+
$ wasm-tools mutate foo.wasm --seed 192 | wasm-tools validate
72+
73+
# Demangle Rust/C++ symbol names in the `name` section, strip all other custom
74+
# sections, and then print out what binary sections remain.
75+
$ wasm-tools demangle foo.wasm | wasm-tools strip | wasm-tools objdump
76+
```
77+
78+
Working with components:
79+
80+
```sh
81+
# Print the WIT interface of a component
82+
$ wasm-tools component wit component.wasm
83+
84+
# Convert WIT text files to a binary-encoded WIT package, printing the result to
85+
# stdout
86+
$ wasm-tools component wit ./wit -t
87+
88+
# Convert a WIT document to JSON
89+
$ wasm-tools component wit ./wit --json
90+
91+
# Round trip WIT through the binary-encoded format to stdout.
92+
$ wasm-tools component wit ./wit --wasm | wasm-tools component wit
93+
94+
# Convert a core WebAssembly binary into a component. Note that this requires
95+
# WIT metadata having previously been embedded in the core wasm module.
96+
$ wasm-tools component new my-core.wasm -o my-component.wasm
97+
98+
# Convert a core WebAssembly binary which uses WASI to a component.
99+
$ wasm-tools component new my-core.wasm -o my-component.wasm --adapt wasi_snapshot_preview1.reactor.wasm
100+
```
101+
102+
### CLI Conventions
103+
104+
There are a few conventions that all CLI commands adhere to:
105+
106+
* All subcommands print "short help" with `-h` and "long help" with `--help`.
107+
* Input is by default read from stdin if no file input is specified (when
108+
applicable).
109+
* Output is by default sent to stdout if a `-o` or `--output` flag is not
110+
provided. Binary WebAssembly is not printed to a tty by default, however.
111+
* Commands which output WebAssembly binaries all support a `-t` or `--wat` flag
112+
to generate the WebAssembly text format instead.
113+
* A `-v` or `--verbose` flag can be passed to enable log messages throughout the
114+
tooling. Verbosity can be turned up by passing the flag multiple times such as
115+
`-vvv`.
116+
* Color in error messages and console output is enabled by default for TTY based
117+
outputs and can be configured with a `--color` argument.
118+
36119
# Tools included
37120

38121
The `wasm-tools` binary internally contains a number of subcommands for working
39-
with wasm modules. Many subcommands also come with Rust crates that can be use
40-
programmatically as well:
122+
with wasm modules and component. Many subcommands also come with Rust crates
123+
that can be use programmatically as well:
41124

42-
| Tool | Crate | Description |
125+
| CLI | Rust Crate | Description |
43126
|------|------|------------|
44127
| `wasm-tools validate` | [wasmparser] | Validate a WebAssembly file |
45128
| `wasm-tools parse` | [wat] and [wast] | Translate the WebAssembly text format to binary |
46129
| `wasm-tools print` | [wasmprinter] | Translate the WebAssembly binary format to text |
47-
| `wasm-tools smith` | [wasm-smith] | Generate a "random" valid WebAssembly module |
130+
| `wasm-tools smith` | [wasm-smith] | Generate a valid WebAssembly module from an input seed |
48131
| `wasm-tools mutate` | [wasm-mutate] | Mutate an input wasm file into a new valid wasm file |
49132
| `wasm-tools shrink` | [wasm-shrink] | Shrink a wasm file while preserving a predicate |
50133
| `wasm-tools dump` | | Print debugging information about the binary format |
@@ -58,6 +141,7 @@ programmatically as well:
58141
| `wasm-tools metadata show` | [wasm-metadata] | Show name and producer metadata in a component or module |
59142
| `wasm-tools metadata add` | | Add name or producer metadata to a component or module |
60143
| `wasm-tools addr2line` | | Translate wasm offsets to filename/line numbers with DWARF |
144+
| `wasm-tools completion` | | Generate shell completion scripts for `wasm-tools` |
61145

62146
[wasmparser]: https://crates.io/crates/wasmparser
63147
[wat]: https://crates.io/crates/wat
@@ -70,9 +154,9 @@ programmatically as well:
70154
[wasm-compose]: https://crates.io/crates/wasm-compose
71155
[wasm-metadata]: https://crates.io/crates/wasm-metadata
72156

73-
The `wasm-tools` CLI is primarily intended to be a debugging aid. The various
74-
subcommands all have `--help` explainer texts to describe more about their
75-
functionality as well.
157+
The `wasm-tools` CLI contains useful tools for debugging WebAssembly modules and
158+
components. The various subcommands all have `--help` explainer texts to
159+
describe more about their functionality as well.
76160

77161
# Libraries
78162

@@ -101,7 +185,16 @@ embedding into a separate project.
101185

102186
# C/C++ bindings
103187

104-
Using the `CMakeLists.txt` in `crates/c-api`, `wasm-tools` can be used from the [`wasm-tools.h` header](crates/c-api/include/wasm-tools.h).
188+
Using the `CMakeLists.txt` in `crates/c-api`, `wasm-tools` can be used from the
189+
[`wasm-tools.h` header](crates/c-api/include/wasm-tools.h). Note that these
190+
bindings do not comprehensively cover all the functionality of this repository
191+
at this time, but please feel free to contribute more if you find functions
192+
useful!
193+
194+
# Contributing
195+
196+
See [CONTRIBUTING.md](./CONTRIBUTING.md) for more information about contributing
197+
to this repository.
105198

106199
# License
107200

0 commit comments

Comments
 (0)