Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 3 additions & 6 deletions .config/dictionaries/project.dic
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ cardano
catid
cbor
cdylib
CFLAGS
chaincode
childs
chksum
Expand Down Expand Up @@ -100,10 +101,7 @@ jsonschema
justfile
keepalives
keypair
Keypair
KEYPAIR
keypairs
Keypairs
libp
libsqlite
libtest
Expand All @@ -125,7 +123,6 @@ mkcron
mkdelay
mkdirat
multiaddr
Multiaddr
multiaddrs
multicodec
multihash
Expand Down Expand Up @@ -164,7 +161,6 @@ pubkey
pwrite
qmark
quic
QUIC
rafal
rankdir
ranksep
Expand Down Expand Up @@ -200,6 +196,7 @@ subpaths
subsec
symlinkat
syscall
sysroot
tailscale
Tdiff
testcov
Expand Down Expand Up @@ -243,4 +240,4 @@ xprivate
xprv
xpub
zilla
zillable
zillable
152 changes: 99 additions & 53 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,83 +5,128 @@
Hermes is a high-performance WebAssembly (WASM) application engine that provides secure,
sandboxed execution of modular applications.

## Quick Start
* [Hermes](#hermes)
* [Prerequisites](#prerequisites)
* [GitHub Token Setup](#github-token-setup)
* [Rust Toolchain Setup](#rust-toolchain-setup)
* [WASI SDK Setup (Local Builds Only)](#wasi-sdk-setup-local-builds-only)
* [Quick Start](#quick-start)
* [Build System](#build-system)
* [Key Commands](#key-commands)
* [Contributing](#contributing)
* [License](#license)

## Prerequisites

Before building Hermes, ensure you have the following installed:

| Tool | Purpose | Installation |
| ------------ | ------------------------------ | ----------------------------------------------------------- |
| **Rust** | Build toolchain | [rustup.rs](https://rustup.rs) |
| **Just** | Command runner | `cargo install just` or `brew install just` |
| **Docker** | Container runtime | [docker.com](https://www.docker.com/get-started) |
| **Earthly** | Containerized builds | [earthly.dev](https://earthly.dev/get-earthly) |
| **LLVM** | WASM target support | `brew install llvm` (macOS) or `apt install llvm` (Linux) |
| **cmake** | Native deps (local builds) | `brew install cmake` (macOS) or `apt install cmake` (Linux) |
| **WASI SDK** | WASM C compiler (local builds) | See [WASI SDK Setup](#wasi-sdk-setup-local-builds-only) |

### GitHub Token Setup

A GitHub token is required for accessing private dependencies:

1. Go to [github.com/settings/tokens](https://github.com/settings/tokens)
2. Generate a new **classic token** with these permissions:
* `public_repo` - Access public repositories
* `read:packages` - Read packages from GitHub Package Registry
3. Create a `.secret` file in the project root (use `.secret.template` as reference):

0. **Github token**:
```bash
cp .secret.template .secret
# Edit .secret and add your token
```

## GitHub Token Setup
### Rust Toolchain Setup

* Go to [github.com/settings/tokens](https://github.com/settings/tokens)
* Generate new classic token with + permissions
* Add to .secret file.
The project uses Rust 1.89 (specified in `hermes/rust-toolchain.toml`).
For local WASM builds, install the required target:

1. **Install Just command runner**:
```bash
rustup install 1.89
rustup target add wasm32-wasip2 --toolchain 1.89
```

```bash
cargo install just
# Or: sudo apt install just
# Or: brew install just
```
### WASI SDK Setup (Local Builds Only)

2. **See all available commands and documentation**:
For local builds (`just build-run-dev-fastest`), the [WASI SDK](https://github.com/WebAssembly/wasi-sdk)
is required to compile C dependencies to WebAssembly.
Skip this if using containerized builds.

```bash
just --list
```
**Download and install** (version 29+):

3. **Build and run:**
<!-- markdownlint-disable MD013 -->
```bash
WASI_VERSION=29
WASI_VERSION_FULL=${WASI_VERSION}.0

```bash
# First time setup (run once)
just check-local-build

# Choose your build approach:
just build-run-dev-fastest # 🚀 Daily dev (local, fastest)
just build-run-dev # 🐳 Team consistency (containerized)
just build-run-all # 📦 Production (full assets)
```
# macOS (Apple Silicon)
curl -LO https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-${WASI_VERSION}/wasi-sdk-${WASI_VERSION_FULL}-arm64-macos.tar.gz
tar xzf wasi-sdk-${WASI_VERSION_FULL}-arm64-macos.tar.gz
sudo mv wasi-sdk-${WASI_VERSION_FULL}-arm64-macos /opt/wasi-sdk

## Build System

This project uses [Just](https://github.com/casey/just) with two build approaches:
# macOS (Intel)
# Use: wasi-sdk-${WASI_VERSION_FULL}-x86_64-macos.tar.gz

| Approach | When to Use | Requirements |
|----------|-------------|-------------|
| 🚀 **Local** | Daily development, rapid iteration | Local Rust + `wasm32-wasip2` |
| 🐳 **Containerized** | Team consistency, CI/CD, final testing | Docker/Podman + Earthly |
# Linux (x86_64)
# Use: wasi-sdk-${WASI_VERSION_FULL}-x86_64-linux.tar.gz

**All detailed documentation is in the `justfile`.** Run `just --list` to see all commands.
# Linux (arm64)
# Use: wasi-sdk-${WASI_VERSION_FULL}-arm64-linux.tar.gz
```

## Key Commands
See [WASI SDK Releases](https://github.com/WebAssembly/wasi-sdk/releases) for all available platforms.

### Build Commands
> **Note:** make sure wasi-sdk is installed to `/opt/wasi-sdk`.

**Main workflows:**
**Set environment variables** (add to `~/.zshrc`, `~/.bashrc`, or `~/.profile`):

* `just build-run-dev-fastest` - 🚀 **Daily development** (local builds, fastest)
* `just build-run-dev` - 🐳 **Team consistency** (containerized, matches CI)
* `just build-run-all` - 📦 **Production** (full assets, deployments)
```bash
export WASI_SDK_PATH="/opt/wasi-sdk"
export CC_wasm32_wasip2="${WASI_SDK_PATH}/bin/clang"
```

**Quick rebuilds:**
Then reload your shell: `source ~/.zshrc` (or restart your terminal)

* `just dev-athena-fast` - WASM only (development)
* `just dev-athena` - WASM only (production)
## Quick Start

**Setup:**
```bash
just check-local-build # Verify your setup (run once)
just build-run-dev # 🐳 Recommended: containerized build (first-time setup)
just build-run-dev-fastest # 🚀 Fast iteration (requires local WASM toolchain)
```

* `just check-local-build` - Verify local Rust (run once)
> **Tip:** Use `just --list` to see all available commands, `just --show <command>` for details.

### Other Commands
## Build System

* `just status` - Show build status
* `just clean-hfs` - Clean application state
* `just --list` - See all available commands
This project uses [Just](https://github.com/casey/just) with two build approaches:

For detailed help: `just --show <command>`
| Approach | When to Use | Requirements |
| ------------------- | -------------------------------------- | ---------------------------- |
| 🐳 **Containerized** | First-time setup, CI/CD, final testing | Docker/Podman + Earthly |
| 🚀 **Local** | Daily development, rapid iteration | Local Rust + `wasm32-wasip2` |

## Development
## Key Commands

For development guidelines, tooling information, and best practices, see DEVELOPMENT.md.
| Command | Description |
| ---------------------------- | ------------------------------------ |
| `just build-run-dev` | 🐳 Containerized build (recommended) |
| `just build-run-dev-fastest` | 🚀 Local build (fast iteration) |
| `just build-run-all` | 📦 Production build (full assets) |
| `just dev-athena-fast` | WASM modules only (dev) |
| `just dev-athena` | WASM modules only (prod) |
| `just check-local-build` | Verify local Rust and WASM toolchain |
| `just status` | Show build status |
| `just clean-hfs` | Clean application state |

## Contributing

Expand All @@ -90,7 +135,8 @@ Please read our [CONTRIBUTING](CONTRIBUTING.md) for guidelines on how to contrib

## License

Licensed under either of [Apache License, Version 2.0](LICENSE-APACHE) or [MIT license](LICENSE-MIT) at your option.
Licensed under either of [Apache License, Version 2.0](LICENSE-APACHE) or
[MIT license](LICENSE-MIT) at your option.

Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in this crate by you, as defined in the Apache-2.0 license, shall
Expand Down
18 changes: 18 additions & 0 deletions hermes/Earthfile
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,24 @@ IMPORT ../wasm/stub-module AS stub-module
# builder : Set up our target toolchains, and copy our files.
builder:
DO rust-ci+SETUP

# Install WASI SDK for compiling C code to wasm32-wasip2
ARG WASI_SDK_VERSION=25
ARG WASI_SDK_VERSION_FULL=${WASI_SDK_VERSION}.0
RUN apt-get update && apt-get install -y wget && \
ARCH=$(dpkg --print-architecture) && \
if [ "$ARCH" = "amd64" ]; then WASI_ARCH="x86_64"; else WASI_ARCH="arm64"; fi && \
wget -q https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-${WASI_SDK_VERSION}/wasi-sdk-${WASI_SDK_VERSION_FULL}-${WASI_ARCH}-linux.tar.gz && \
tar xzf wasi-sdk-${WASI_SDK_VERSION_FULL}-${WASI_ARCH}-linux.tar.gz && \
mv wasi-sdk-${WASI_SDK_VERSION_FULL}-${WASI_ARCH}-linux /opt/wasi-sdk && \
rm wasi-sdk-${WASI_SDK_VERSION_FULL}-${WASI_ARCH}-linux.tar.gz && \
apt-get clean && rm -rf /var/lib/apt/lists/*

# Configure environment for wasm32-wasip2 C compilation
ENV WASI_SDK_PATH="/opt/wasi-sdk"
ENV CC_wasm32_wasip2="${WASI_SDK_PATH}/bin/clang"
ENV CFLAGS_wasm32_wasip2="--sysroot=${WASI_SDK_PATH}/share/wasi-sysroot"

COPY --dir .cargo .config bin apps schemas .
COPY Cargo.toml .
COPY clippy.toml deny.toml rustfmt.toml .
Expand Down
17 changes: 17 additions & 0 deletions hermes/apps/athena/Earthfile
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,23 @@ builder:

DO rust-ci+SETUP

# Install WASI SDK for compiling C code to wasm32-wasip2
ARG WASI_SDK_VERSION=25
ARG WASI_SDK_VERSION_FULL=${WASI_SDK_VERSION}.0
RUN apt-get update && apt-get install -y wget && \
ARCH=$(dpkg --print-architecture) && \
if [ "$ARCH" = "amd64" ]; then WASI_ARCH="x86_64"; else WASI_ARCH="arm64"; fi && \
wget -q https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-${WASI_SDK_VERSION}/wasi-sdk-${WASI_SDK_VERSION_FULL}-${WASI_ARCH}-linux.tar.gz && \
tar xzf wasi-sdk-${WASI_SDK_VERSION_FULL}-${WASI_ARCH}-linux.tar.gz && \
mv wasi-sdk-${WASI_SDK_VERSION_FULL}-${WASI_ARCH}-linux /opt/wasi-sdk && \
rm wasi-sdk-${WASI_SDK_VERSION_FULL}-${WASI_ARCH}-linux.tar.gz && \
apt-get clean && rm -rf /var/lib/apt/lists/*

# Configure environment for wasm32-wasip2 C compilation
ENV WASI_SDK_PATH="/opt/wasi-sdk"
ENV CC_wasm32_wasip2="${WASI_SDK_PATH}/bin/clang"
ENV CFLAGS_wasm32_wasip2="--sysroot=${WASI_SDK_PATH}/share/wasi-sysroot"

COPY wasm+wasi-src/wasi $wasi_src_dir

# check-shared : runs native unit tests of the shared crate.
Expand Down
2 changes: 1 addition & 1 deletion hermes/apps/athena/modules/doc-sync/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ All heavy operations are delegated to the host-side implementation for performan
The module exposes an HTTP endpoint for testing:

```bash
curl -X POST http://localhost:5000/api/doc-sync/post \
curl -X POST http://localhost:7878/api/doc-sync/post \
-H "Host: athena.hermes.local" \
-H "Content-Type: text/plain" \
-d "Hello, IPFS!"
Expand Down
2 changes: 1 addition & 1 deletion hermes/apps/athena/modules/doc-sync/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ fn get_document_by_cid(cid: &[u8]) -> anyhow::Result<Option<Vec<u8>>> {
///
/// Example:
/// ```bash
/// curl -X POST http://localhost:5000/api/doc-sync/post \
/// curl -X POST http://localhost:7878/api/doc-sync/post \
/// -H "Host: athena.hermes.local" \
/// -H "Content-Type: text/plain" \
/// -d "Hello, IPFS!"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ use tracing::{debug, error, info};

use super::routing::router;

/// HTTP Gateway default port
const DEFAULT_GATEWAY_PORT: u16 = 5000;
/// HTTP Gateway default port (`7878` avoids conflicts with macOS `AirPlay` on `5000`)
const DEFAULT_GATEWAY_PORT: u16 = 7878;

/// hostname (node name)
#[cfg_attr(debug_assertions, derive(Debug))]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ pub(crate) fn host_resolver(headers: &HeaderMap) -> anyhow::Result<(ApplicationN
.and_then(|h| h.to_str().ok())
.ok_or_else(|| anyhow!("Missing or invalid Host header"))?;

// Strip port if present (e.g., "app.hermes.local:5000" -> "app.hermes.local")
// Strip port if present (e.g., "app.hermes.local:7878" -> "app.hermes.local")
let host_without_port = host.split(':').next().unwrap_or(host);

// Parse <app.name>.hermes.local format
Expand Down
2 changes: 1 addition & 1 deletion hermes/bin/tests/integration/tests/serial/staked_ada.rs
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ fn staked_ada_requests() -> anyhow::Result<()> {
std::thread::sleep(Duration::from_secs(10));

let url_builder = |stake_address: &str| {
format!("http://localhost:5000/api/gateway/v1/cardano/assets/{stake_address}")
format!("http://localhost:7878/api/gateway/v1/cardano/assets/{stake_address}")
};
// cspell:disable
let url_with_mocked_address =
Expand Down
4 changes: 2 additions & 2 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -260,7 +260,7 @@ clean-hfs:
fi

# Run the Athena application
# Test with: curl -H "Host: app.hermes.local" http://localhost:5000/api/gateway/v1/rbac/registration
# Test with: curl -H "Host: app.hermes.local" http://localhost:7878/api/gateway/v1/rbac/registration
run-athena:
#!/usr/bin/env bash
set -euo pipefail
Expand Down Expand Up @@ -289,7 +289,7 @@ run-athena:
echo " Allowed Path Prefixes: $REDIRECT_ALLOWED_PATH_PREFIXES"
echo ""
echo "🌐 Starting HTTP server..."
echo "💡 Test with: curl -H 'Host: app.hermes.local' http://localhost:5000/api/gateway/v1/rbac/registration"
echo "💡 Test with: curl -H 'Host: app.hermes.local' http://localhost:7878/api/gateway/v1/rbac/registration"
echo "🛑 Press Ctrl+C to stop"
echo ""

Expand Down
20 changes: 17 additions & 3 deletions wasm/wasi/Earthfile
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,23 @@ builder:

DO rust-ci+SETUP

# Install WASI SDK for compiling C code to wasm32-wasip2
ARG WASI_SDK_VERSION=25
ARG WASI_SDK_VERSION_FULL=${WASI_SDK_VERSION}.0
RUN apt-get update && apt-get install -y wget && \
ARCH=$(dpkg --print-architecture) && \
if [ "$ARCH" = "amd64" ]; then WASI_ARCH="x86_64"; else WASI_ARCH="arm64"; fi && \
wget -q https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-${WASI_SDK_VERSION}/wasi-sdk-${WASI_SDK_VERSION_FULL}-${WASI_ARCH}-linux.tar.gz && \
tar xzf wasi-sdk-${WASI_SDK_VERSION_FULL}-${WASI_ARCH}-linux.tar.gz && \
mv wasi-sdk-${WASI_SDK_VERSION_FULL}-${WASI_ARCH}-linux /opt/wasi-sdk && \
rm wasi-sdk-${WASI_SDK_VERSION_FULL}-${WASI_ARCH}-linux.tar.gz && \
apt-get clean && rm -rf /var/lib/apt/lists/*

# Configure environment for wasm32-wasip2 C compilation
ENV WASI_SDK_PATH="/opt/wasi-sdk"
ENV CC_wasm32_wasip2="${WASI_SDK_PATH}/bin/clang"
ENV CFLAGS_wasm32_wasip2="--sysroot=${WASI_SDK_PATH}/share/wasi-sysroot"

COPY ..+wasi-src/wasi $wasi_src_dir

# Build the WASI Docs:
Expand Down Expand Up @@ -103,9 +120,6 @@ BUILD_RUST_COMPONENT:
COPY --keep-ts --dir src .
COPY Cargo.toml .

# Fix missing header error when compiling zstd while building rbac-registration
ENV CFLAGS_wasm32_wasip2="--sysroot=/usr/aarch64-linux-gnu"

DO rust-ci+CARGO \
--args "build --target wasm32-wasip2 --release" \
--output="wasm32-wasip2/release/$out"
Expand Down