Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2,558 changes: 413 additions & 2,145 deletions Cargo.lock

Large diffs are not rendered by default.

10 changes: 5 additions & 5 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ anyhow = "1"
base64 = "0.22"
bytes = "1.8"
docker_credential = "1.2.1"
etcetera = "0.8"
etcetera = "0.10"
futures-util = "0.3.30"
oci-client = { version = "0.14", default-features = false, features = [
"rustls-tls",
Expand All @@ -27,7 +27,7 @@ serde_json = "1"
sha2 = "0.10"
tempfile = "3.10.1"
testcontainers = { version = "0.23" }
thiserror = "1.0"
thiserror = "2.0"
tokio = "1.35.1"
tokio-util = "0.7.10"
toml = "0.8"
Expand All @@ -39,6 +39,6 @@ tracing-subscriber = { version = "0.3.18", default-features = false, features =
wasm-pkg-common = { version = "0.10.0", path = "crates/wasm-pkg-common" }
wasm-pkg-client = { version = "0.10.0", path = "crates/wasm-pkg-client" }
wasm-pkg-core = { version = "0.10.0", path = "crates/wasm-pkg-core" }
wasm-metadata = "0.224"
wit-component = "0.224"
wit-parser = "0.224"
wasm-metadata = "0.226"
wit-component = "0.226"
wit-parser = "0.226"
50 changes: 10 additions & 40 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
Tools to package up [Wasm Components](https://github.com/webassembly/component-model)

This repo contains several Rust crates that can be used for fetching and publishing Wasm Components
to OCI or Warg registries. It is also the home of the `wkg` command line tool, which exposes all the
to OCI registries. It is also the home of the `wkg` command line tool, which exposes all the
functionality of the libraries. The first (but not only) focus of this project is allow for fetching
of Wit Interfaces stored as components for use in creating components. It can also be used to fetch
and publish component "libraries" to/from a registry.
Expand Down Expand Up @@ -94,24 +94,9 @@ another = { registry = "another", metadata = { preferredProtocol = "oci", "oci"
# Same as namespace_registries above, but for a specific package.
"example:bar" = { registry = "another", metadata = { preferredProtocol = "oci", "oci" = {registry = "ghcr.io", namespacePrefix = "webassembly/" } } }

# This section contains a mapping of registries to their configuration. There are currently 3
# supported types of registries: "oci", "warg", and "local". The "oci" type is the default. The
# example below shows a use case that isn't yet super common (registries that speak multiple protocols)
# but is included for completeness.
# This section contains a mapping of registries to their configuration. There are currently 2
# supported types of registries: "oci" and "local". The "oci" type is the default.
[registry."acme.registry.com"]
# This field is only required if more that one protocol is supported. It indicates which protocol
# to use by default. If this is not set, then the fallback (oci) will be used.
default = "warg"
[registry."acme.registry.com".warg]
# A path to a valid warg config file. If this is not set, the `wkg` CLI (but not the libraries)
# will attempt to load the config from the default location(s).
config_file = "/a/path"
# An optional authentication token to use when authenticating with a registry.
auth_token = "an-auth-token"
# An optional key for signing the component. Ideally, you should just let warg use the keychain
# or programmatically set this key in the config without writing to disk. This offers an escape
# hatch for when you need to use a key that isn't in the keychain.
signing_key = "ecdsa-p256:2CV1EpLaSYEn4In4OAEDAj5O4Hzu8AFAxgHXuG310Ew="
[registry."acme.registry.com".oci]
# The auth field can either be a username/password pair, or a base64 encoded `username:password`
# string. If no auth is set, the `wkg` CLI (but not the libraries) will also attempt to load the
Expand All @@ -131,11 +116,11 @@ root = "/a/path"
# If a registry only has a config section for one protocol, then that protocol is automatically
# the default. The following is equivalent to:
# [registry."example.com"]
# default = "warg"
# [registry."example.com".warg]
# config_file = "/a/path"
[registry."example.com".warg]
config_file = "/a/path"
# default = "oci"
# [registry."example.com".oci]
# auth = { username = "open", password = "sesame" }
[registry."example.com".oci]
auth = { username = "open", password = "sesame" }

# Configuration for the "another" registry defined above.
[registry."another".oci]
Expand All @@ -155,24 +140,19 @@ A full example of what this `registry.json` file should look like is below:

```json
{
"preferredProtocol":"warg",
"warg": {"url":"https://warg.example.com"},
"preferredProtocol":"oci",
"oci": {"registry": "ghcr.io", "namespacePrefix": "webassembly/"}
}
```

The `preferredProtocol` field is optional and specifies which protocol the registry expects you to
use in the case where it supports both OCI and Warg. If both `warg` and `oci` config is in the
`registry.json` it is _highly recommended_ that this field be set.
use. While this field is present for future compatibility, it's generally fixed to "oci" in this implementation.

For the `oci` config, the `registry` field is the base URL of the OCI registry, and the
`namespacePrefix` field is the prefix that is used to store components in the registry. So in the
example above (which is for wasi.dev), the components will be available at
`ghcr.io/webassembly/$NAMESPACE/$PACKAGE:$VERSION` (e.g. `ghcr.io/webassembly/wasi/http:0.2.1`).

For the `warg` config, the `url` field is the base URL of the Warg registry used when connecting the
client. Namespacing for warg is built in to the protocol.

Please note that for backwards compatibility, with previous tooling and versions of the `wkg` tool,
you may also encounter a `registry.json` file that looks different. These files are still supported,
but should be considered deprecated.
Expand All @@ -186,16 +166,6 @@ For OCI registries, the JSON looks like this:
}
```



For Warg registries, the JSON looks like this:

```json
{
"wargUrl": "https://warg.wa.dev"
}
```

### Conventions for storing components in OCI

Astute observers will note that OCI requires a specific structure for how those components are
Expand Down
3 changes: 0 additions & 3 deletions crates/wasm-pkg-client/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,7 @@ toml = { workspace = true }
tracing = { workspace = true }
tracing-subscriber = { workspace = true }
url = "2.5.0"
warg-client = "0.9.2"
warg-crypto = "0.9.2"
wasm-metadata = { workspace = true }
warg-protocol = "0.9.2"
wasm-pkg-common = { workspace = true, features = ["metadata-client", "tokio"] }
wit-component = { workspace = true }

Expand Down
6 changes: 1 addition & 5 deletions crates/wasm-pkg-client/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,6 @@ pub mod local;
pub mod oci;
mod publisher;
mod release;
pub mod warg;

use std::path::Path;
use std::sync::Arc;
Expand All @@ -55,7 +54,7 @@ pub use wasm_pkg_common::{
};
use wit_component::DecodedWasm;

use crate::{loader::PackageLoader, local::LocalBackend, oci::OciBackend, warg::WargBackend};
use crate::{loader::PackageLoader, local::LocalBackend, oci::OciBackend};

pub use release::{Release, VersionInfo};

Expand Down Expand Up @@ -276,9 +275,6 @@ impl Client {
&registry_config,
&registry_meta,
)?),
"warg" => {
Box::new(WargBackend::new(&registry, &registry_config, &registry_meta).await?)
}
other => {
return Err(Error::InvalidConfig(anyhow!(
"unknown backend type {other:?}"
Expand Down
13 changes: 10 additions & 3 deletions crates/wasm-pkg-client/src/oci/loader.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
use async_trait::async_trait;
use futures_util::{StreamExt, TryStreamExt};
use oci_client::{manifest::OciDescriptor, RegistryOperation};
use warg_protocol::Version;
use wasm_pkg_common::{package::PackageRef, Error};
use wasm_pkg_common::{
package::{PackageRef, Version},
Error,
};

use crate::{
loader::PackageLoader,
Expand Down Expand Up @@ -36,7 +38,12 @@ impl PackageLoader for OciBackend {
yanked: false,
}),
Err(err) => {
tracing::warn!(?tag, error = ?err, "Ignoring invalid version tag");
// Signature tags all start with a SHA and shouldn't generate a warning
if tag.starts_with("sha256-") {
tracing::debug!(?tag, "Ignoring signature tag");
} else {
tracing::warn!(?tag, error = ?err, "Ignoring invalid version tag");
}
None
}
})
Expand Down
2 changes: 1 addition & 1 deletion crates/wasm-pkg-client/src/oci/publisher.rs
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ impl PackagePublisher for OciBackend {
homepage.to_string(),
);
}
if let Some(authors) = &meta.author {
if let Some(authors) = &meta.authors {
annotations.insert(
"org.opencontainers.image.authors".to_string(),
authors.to_string(),
Expand Down
Loading