Merge branch 'main' into docs/prereqs-rust

Author: mkatychev

component-model/src/SUMMARY.md

@@ -25,7 +25,7 @@
   - [Authoring Components](./creating-and-consuming/authoring.md)
   - [Composing Components](./creating-and-consuming/composing.md)
   - [Running Components](./creating-and-consuming/running.md)
-  - [Distributing Components](./creating-and-consuming/distributing.md)
+  - [Distributing and Fetching Components and WIT](./creating-and-consuming/distributing.md)
 - [Tutorial](./tutorial.md)
 
 # Runtime Support

component-model/src/creating-and-consuming/distributing.md

@@ -1,7 +1,158 @@
-# Distributing Components
+# Distributing and Fetching Components and WIT
 
 Modern applications rely extensively on third-party packages - so extensively that distributing packages is almost an industry in itself. Traditionally, these have been specific to a language. For example, JavaScript developers are used to using packages from NPM, and Rust developers use `crates.io`. Some runtimes support binary distribution and linking, enabling limited cross-language interop; for example, Maven packages can be written in any language that targets the Java runtime. Services like this are variously referred to as "package managers" or "registries."
 
-Publishing and distribution are not defined by the core component model, but will form an important part of the component ecosystem. For example, if you're writing JavaScript, and want to pull in a highly optimised machine learning algorithm written in C and compiled to Wasm, you should be able to invoke it from a registry, just as easily as you would add a NPM package from the NPM registry.
+Publishing and distribution are not defined by the core component model, but form important part of the component ecosystem. For example, if you're writing JavaScript, and want to pull in a highly optimised machine learning algorithm written in C and compiled to Wasm, you can pull it from a registry, ideally just as easily as you would add a NPM package from the NPM registry.
 
-Publishing and distribution is a work in progress. The proposed registry protocol is [warg](https://warg.io/), but this is still in development, and there are no public warg registries as yet. You can find more information about the development of the registry protocol [here](https://github.com/bytecodealliance/governance/blob/main/SIGs/SIG-Registries/proposal.md).
+You can get involved with improving the packaging and hosting of Wasm components by joining the [Bytecode Alliance Packaging Special Interest Group (SIG)](https://github.com/bytecodealliance/governance/blob/main/SIGs/sig-packaging/proposal.md).
+
+## The `wkg` Registry Tool
+
+The [`wasm-pkg-tools` project](https://github.com/bytecodealliance/wasm-pkg-tools) enables fetching and publishing Wasm components to OCI registries. It contains a `wkg` CLI tool that eases distributing and fetching components and WIT packages. The usual way of using `wkg` is to address packages by their package name, i.e. `example:[email protected]`. When using `wkg` this way, you don't need to know about the physical location of the package, as the `wkg` configuration handles that for you. If you need to, though, you can also use `wkg` to work with OCI artifacts directly, addressing them by OCI references when using the `wkg oci` subcommand.
+
+`wkg` contains several subcommand:
+
+- `wkg oci` - pushes/pulls Wasm artifacts to/from any OCI registry
+- `wkg publish` - publish components or WIT packages by package name
+- `wkg get` - pulls components or WIT packages by package name
+- `wkg wit` - commands for interacting with WIT files and dependencies
+- `wkg config` - interact with the `wkg` configuration
+
+The following sections detail a subset of actions that can be performed with `wkg`.
+
+## `wkg` Configuration Files
+
+When you use most `wkg` commands (`wkg oci` being the exception), you don't interact with physical locations, only with package names. The `wkg` configuration file is used to map package naming to physical location. It provides the ability to configure:
+
+- The default registry for packages in a given namespace. For example, the location for `wasi` packages such as `wasi:clocks` or `wasi:http`.  
+- Registry overrides for specific packages, or packages not stored in the same place as the rest of their namespace. For example, if `wasi:key-value` were stored in a different registry from other `wasi` packages.  
+- The default registry for all packages not listed in one of the previous sections  
+
+The configuration file also includes credentials for private registries, or for pushing to registries where you have permission, and other configuration options.  See the [`wkg` docs for more configuration options](https://github.com/bytecodealliance/wasm-pkg-tools?tab=readme-ov-file#configuration).
+
+For example, to fetch WASI packages, such as `wasi:clocks` and `wasi:http`, you can add a line under the `namespace_registries` section for the `wasi` namespace. Specifically, the example below configures `wkg` to fetch WASI packages from the [WebAssembly OCI GitHub Container Registry](https://github.com/orgs/WebAssembly/packages), where the latest interfaces are published upon WASI releases. To edit your `wkg` config file, run `wkg config --edit`. 
+
+> Remember, all package names consist of the a namespace field followed by the package field. The package name `wasi:clocks` has a namespace of `wasi` and package field of `clocks`. In this way, the following configuration ensures that `wkg` will know to route fetches and publishes of any `wasi:<package field>` to the configured location.
+
+```toml
+# $XDG_CONFIG_HOME/wasm-pkg/config.toml
+default_registry = "ghcr.io"
+
+[namespace_registries]
+# Tell wkg that packages with the `wasi` namespace are in an OCI registry under ghcr.io/webassembly 
+wasi = { registry = "wasi",  metadata = { preferredProtocol = "oci", "oci" = {registry = "ghcr.io", namespacePrefix = "webassembly/" } } }
+```
+
+As a more generic example, The following configuration, instructs `wkg` to use [ttl.sh](https://ttl.sh/) OCI registry for all packages with the `docs` namespace.
+
+```toml
+# $XDG_CONFIG_HOME/wasm-pkg/config.toml
+default_registry = "ghcr.io"
+
+[namespace_registries]
+# Instruct wkg to use the OCI protocol to fetch packages with the `foo` namespace from ttl.sh/wasm-components
+docs = { registry = "docs-registry",  metadata = { preferredProtocol = "oci", "oci" = {registry = "ttl.sh", namespacePrefix = "wasm-components/" } } }
+```
+
+> Note: the registry name can be referenced in the `package_registry_overrides` section of the `wkg` config to provide overrides for specific packages of a namespace.
+
+## Distributing WIT and Components by Package Name with `wkg publish`
+
+Once you've [configured `wkg`](#wkg-configuration-files) to know where to publish packages to, you can use the `wkg publish` command to publish *components* or *interfaces* to be consumed by others.
+
+Imagine you have defined the following `adder` world in WIT:
+
+```wit
+package docs:[email protected];
+
+interface add {
+    add: func(a: u32, b: u32) -> u32;
+}
+
+world adder {
+    export add;
+}
+```
+
+You can publish this *WIT* using `wkg` by wrapping it up as a Wasm component. Yes, you heard that right! We are packaging WIT as Wasm.
+
+```sh
+# Package the contents of add WIT directory as Wasm
+wkg wit build --wit-dir tutorial/wit/adder
+# Publish the produced component
+wkg publish docs:[email protected]
+```
+
+If you had configured `wkg` as described in the [`wkg` configuration section](#wkg-configuration-files), this would publish the component to `ttl.sh/wasm-components/docs/adder:0.1.0`. This WIT can then be fetched using `wkg get`, specifying the format `wit`:
+
+```sh
+wkg get --format wit docs:[email protected] --output adder.wit
+```
+
+Instead of publishing the WIT interface, you could publish the built component by running:
+
+```sh
+wkg publish adder.wasm --package docs:[email protected]
+```
+
+You can then fetch the component by running:
+
+```sh
+wkg get docs:[email protected] --output adder.wasm
+```
+
+## More Generic Operations with `wkg oci`
+
+The `wkg oci` subcommand enables pushing/pulling Wasm artifacts to/from any OCI registry. Unlike `wkg publish` and `wkg get`, providing the WIT package is not required.
+
+To push a component to an OCI registry, use `wkg oci pull`. The example below pushes a component to a GitHub Container Registry.  
+
+```sh
+wkg oci push ghcr.io/user/component:0.1.0 component.wasm
+```
+
+To pull a component, run:
+
+```sh
+wkg oci pull ghcr.io/user/component:0.1.0 -o component.wasm
+```
+
+## Fetching WIT Package Dependencies using `wkg`
+
+Sometimes fetching a single package is not sufficient because it depends on other packages. For example, the following world describes a simple Wasm service which requires `wasi:http/proxy`:
+
+```wit
+package foo:wasi-http-service;
+
+world target-world {
+  include wasi:http/[email protected];
+}
+```
+
+You may be tempted to simply get the `wasi:http` package with `wkg get  --format wit wasi:[email protected] -o wit/deps/http/`. However, `wasi:http` depends on other WASI packages such as `wasi:clocks` and `wasi:io`. To make sure to fetch a package and all its dependencies, use `wkg wit fetch`, which will read the package containing the world(s) you have defined in the given wit directory (`wit` by default). It will then fetch the
+dependencies and write them to the `deps` directory along with a lock file.
+
+After placing the above file in `./wit`, run the following to fetch the dependencies:
+
+```sh
+wkg wit fetch
+```
+
+The `./wit` directory will be populated as follows:
+```sh
+wit
+├── deps
+│   ├── wasi-cli-0.2.3
+│   │   └── package.wit
+│   ├── wasi-clocks-0.2.3
+│   │   └── package.wit
+│   ├── wasi-http-0.2.3
+│   │   └── package.wit
+│   ├── wasi-io-0.2.3
+│   │   └── package.wit
+│   └── wasi-random-0.2.3
+│       └── package.wit
+└── world.wit
+```
+
+Now, you can use the language toolchain of your choice to generate bindings and create your component.

component-model/src/language-support/go.md

@@ -286,6 +286,8 @@ Under the hood, TinyGo invokes `wasm-tools` to embed the WIT file to the module
 tinygo build -target=wasip2 -o add.wasm --wit-package docs:[email protected] --wit-world adder main.go
 ```
 
+> **WARNING:** By default, tinygo includes all debug-related information in your .wasm file. That is desirable when prototyping or testing locally to obtain useful backtraces in case of errors (for example, with `wasmtime::WasmBacktraceDetails::Enable`). To remove debug data and optimize your binary file, build with `-no-debug`. The resulting .wasm file will be considerably smaller (up to 75% reduction in size).
+
 We now have an add component that satisfies our `adder` world, exporting the `add` function, which
 
 We can confirm using the `wasm-tools component wit` command:

component-model/src/language-support/javascript.md

@@ -106,6 +106,8 @@ You should see output like the following:
 OK Successfully written add.wasm.
 ```
 
+> **WARNING:** By using `--disable all`, your component won't get access to any WASI interfaces that might be useful for debugging or logging. For example, you can't `console.log(...)` or `console.error(...)` without `stdio`; you can't use `Math.random()` without `random`; and you can't use `Date.now()` or `new Date()` without `clocks`. Please note that calls to `Math.random()` or `Date.now()` will return seemingly valid outputs, but without actual randomness or timestamp correctness.
+
 [mdn-js-module]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules
 
 ## Running the Component in the `example-host`

component-model/src/language-support/rust.md

@@ -8,43 +8,43 @@ the component's implementation language.
 > [!NOTE]
 > You can find more details about `cargo-component` on [crates.io](https://crates.io/crates/cargo-component).
 
-## Setup
+## 1. Setup
 
 Install [`cargo-component`][cargo-component-install]:
-```sh
+```console
 cargo install --locked cargo-component
 ```
 Install [`wasm-tools`](https://github.com/bytecodealliance/wasm-tools#installation):
-```sh
+```console
 cargo install --locked wasm-tools
 ```
 Install [`wasmtime`](https://github.com/bytecodealliance/wasmtime#installation):
-```sh
+```console
 curl https://wasmtime.dev/install.sh -sSf | bash
 ```
 Clone the [component-docs](https://github.com/bytecodealliance/component-docs) repo:
-```sh
+```console
 git clone https://github.com/bytecodealliance/component-docs
 ```
 
-## Scaffolding a Component
+## 2. Scaffolding a Component
 
 We will create a component in Rust that implements the `add` function exported
-by the [`docs:adder/adder`][docs-adder] world in the
+by the [`adder` world][docs-adder] world in the
 `docs:adder`
 [package](https://github.com/WebAssembly/component-model/blob/main/design/mvp/WIT.md#package-names).
 
 First `cd` into the `tutorial` directory found in the repo we just cloned:
-```sh
+```console
 cd component-docs/component-model/examples/tutorial
 ```
 
 Now create a new WebAssembly component package called `add`:
-```sh
+```console
 cargo component new add --lib && cd add
 ```
 
-## Adding the WIT world
+## 3. Adding the WIT world
 
 We now need to change our generated `wit/world.wit` to match `docs:adder`:
 ```wit
@@ -59,20 +59,20 @@ to the following:
 package = "docs:adder"
 ```
 
-## Generating bindings
+## 4. Generating bindings
 
 Now that we've updated our `world.wit` and `Cargo.toml`, we can re-generate
 bindings with the command below:
 
-```sh
+```console
 cargo component bindings
 ```
 
 `cargo-component` will generate bindings for our
 world and create a `Guest` trait that a component should
-implement
+implement.
 
-## Implementing the `Guest` trait
+## 5. Implementing the `Guest` trait
 
 Implement the `Guest` trait in `src/lib.rs`, using the scaffolded code. Your
 code should look something like the following:
@@ -81,17 +81,20 @@ code should look something like the following:
 {{#include ../../examples/tutorial/adder/src/lib.rs}}
 ```
 
-## Building a Component
+## 6. Building a Component
 
 Now, let's build our component, being sure to optimize with a release build:
 
-```sh
+```console
 cargo component build --release
 ```
 
 You can use `wasm-tools` to inspect the WIT package generated by `cargo-component`:
 
-```sh
+
+You can use `wasm-tools component wit` to output the WIT package of the component:
+
+```console
 wasm-tools component wit target/wasm32-wasip1/release/add.wasm
 ```
 
@@ -110,6 +113,8 @@ package docs:[email protected] {
 }
 ```
 
+> **WARNING:** Building with `--release` removes all debug-related information from the resulting .wasm file. When prototyping or testing locally, you might want to avoid `--release` to obtain useful backtraces in case of errors (for example, with `wasmtime::WasmBacktraceDetails::Enable`). Note: the resulting .wasm file will be considerably larger (likely 4MB+).
+
 ### Running a Component
 
 To verify that our component works, lets run it from a Rust application that knows how to run a
@@ -120,7 +125,7 @@ Rust bindings, bring in WASI worlds, and execute the component.
 
 ```sh
 $ cd examples/example-host
-$ cargo run --release -- 1 2 ../add/target/wasm32-wasip1/release/add.wasm
+$ cargo run --release -- 1 2 ../add/target/wasm32-wasip1/release/adder.wasm
 1 + 2 = 3
 ```
 
@@ -131,8 +136,8 @@ It's often preferable to export an interface rather than a function, either to
 comply with an existing specification or to capture several functions and types
 at once.
 
-For example, to implement the [`docs:adder/adder`](#adding-the-wit-world)
-world, you would write the following Rust code:
+For example, to implement the [`adder` world](#adding-the-wit-world), you would
+write the following Rust code:
 
 ```rust
 {{#include ../../examples/tutorial/adder/src/lib.rs}}
@@ -226,7 +231,8 @@ world root {
 }
 ```
 
-As the import is unfulfilled, the `calculator.wasm` component could not run by itself in its current form. To fulfill the `add` import, so that only `calculate` is exported, you would need to [compose the `calculator.wasm` with some `exports-add.wasm` into a single, self-contained component](../creating-and-consuming/composing.md).
+As the import is unfulfilled, the `calculator.wasm` component could not run by itself in its current form. To fulfill the `add` import, so that
+only `calculate` is exported, you would need to [compose the `calculator.wasm` with some `adder.wasm` into a single, self-contained component](../creating-and-consuming/composing.md).
 
 ## Creating a command component with `cargo component`