Skip to content

sarus-suite/cluster-tooling

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

132 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Sarus Suite's Cluster Tooling Super-Crate Workspace

This repository is a Rust workspace (“super-crate”) that brings together the main components of the Sarus Suite for the cluster tooling so they can be developed and built from a single checkout. The workspace is intended to make it easy to build tooling artifacts like skybox (a Slurm SPANK plugin written in Rust) inside a reproducible environment that already contains the required system dependencies such as Slurm headers.

Instead of consuming raster and podman-driver directly from git, this workspace patches those dependencies to local paths, allowing you to iterate on all components together.


Workspace Layout

.
├── crates/
│   ├── skybox/            # Slurm SPANK plugin (cdylib)
│   ├── sarusctl/          # Test CLI and utilities
│   ├── podman-driver/     # sarus-suite-podman-driver crate
│   └── raster/            # EDF rendering / validation library
├── .devcontainer/         # Root devcontainer with Rust + Slurm headers
└── Cargo.toml             # Workspace manifest (super-crate)

Members

Crate Purpose
skybox Slurm SPANK plugin implemented in Rust, built as a shared library
sarusctl CLI for testing raster + podman integration
sarus-suite-podman-driver Library for constructing Podman invocations from EDF
raster EDF schema, rendering, and validation utilities

Local Development Model

Downstream crates declare dependencies using git URLs:

raster = { git = "https://github.com/sarus-suite/raster" }
sarus-suite-podman-driver = { git = "https://github.com/sarus-suite/podman-driver" }

The workspace root overrides these with path patches so that local versions are used:

[patch."https://github.com/sarus-suite/podman-driver"]
sarus-suite-podman-driver = { path = "crates/podman-driver" }

[patch."https://github.com/sarus-suite/raster"]
raster = { path = "crates/raster" }

This allows:

  • editing raster or podman-driver
  • immediately rebuilding skybox
  • without publishing to git or modifying individual crates

Devcontainer (Recommended Build Environment)

The repository includes a root devcontainer called:

skybox-dev (rust + slurm-dev)

What the container provides

  • openSUSE Leap base image
  • Rust toolchain via rustup
  • gcc/clang, cmake, pkg-config, and build essentials
  • Slurm 24.05.x headers including SPANK
  • Environment configured with:
CPATH=/usr/local/include:/usr/local/include/slurm:/usr/include

Starting the Devcontainer

Using the Devcontainer CLI

From the repository root:

devcontainer up --workspace-folder . --config .devcontainer/opensuse/devcontainer.json
devcontainer exec --workspace-folder . --config .devcontainer/opensuse/devcontainer.json -- bash

You are now inside the prepared build environment.

VS Code

Open the repository in VS Code and choose:

Reopen in Container

using the root devcontainer configuration.


Building Skybox

All commands should be executed from the workspace root inside the devcontainer.

Clean + update (optional but recommended after changes)

cargo clean
cargo update

Build only skybox

cargo build -p skybox

Release build

cargo build -p skybox --release

The result is a shared library under:

target/debug/   (or target/release/)

Building from a subdirectory

Always run cargo commands from the workspace root. Running cargo build inside crates/skybox can bypass workspace configuration.


Typical Build Workflow

  1. Start devcontainer
devcontainer up --workspace-folder .
devcontainer exec --workspace-folder . -- bash
  1. Edit:

    • raster schema
    • podman-driver logic
    • skybox plugin
  2. Rebuild skybox:

cargo build -p skybox
  1. Rebuild devcontainer in case of devcontainer/Dockerfile changes
  • Edit .devcontainer/Dockerfile or .devcontainer/devcontainer.json
  • Get the container ID of the devcontainer with devcontainer up --workspace-folder .
  • docker stop <container ID>
  • docker rm <container ID>
  • Restart devcontainer as in 1

Development Workflows

Upstream remotes

The best way to work with upstream repos is to set short remote names, for every crate do crate-upstream. Example:

git remote add skybox-upstream https://github.com/sarus-suite/skybox.git

Keeping workspace and subtrees up to date (before dev work)

git switch main
git pull --ff-only origin main
# Update all remotes
git fetch --all --prune

# update each subtree into a squashed commit (this keep things simple)
git subtree pull --prefix=crates/raster        raster-upstream        main --squash
git subtree pull --prefix=crates/podman-driver podman-driver-upstream main --squash
git subtree pull --prefix=crates/sarusctl      sarusctl-upstream      main --squash
git subtree pull --prefix=crates/skybox        skybox-upstream        main --squash

Work on a new feature

Create new feature branch in workspace

git switch -c <feature branch name>

Do your work

# Do your code edit
vi cargo/skybox/src/xyz

# Build and test in dev container
cargo build -p skybox
cargo test

commit to workspace

git add ...
git commit -m "feature work ..."

Export each modified subtree and push to upstream crate

git subtree split --prefix=crates/skybox -b skybox-export

This will add a history changes into a branch that only contains the changes related crates/skybox

Then we push that to upstream as a new branch

git push skybox-upstream skybox-export:my-new-skybox-branch-for-mr

The same pattern can be repeated for all other crates.

Reset workspace

git switch main
git pull --ff-only origin main
# Update all remotes
git fetch --all --prune

# update each subtree into a squashed commit (this keep things simple)
git subtree pull --prefix=crates/raster        raster-upstream        main --squash
git subtree pull --prefix=crates/podman-driver podman-driver-upstream main --squash
git subtree pull --prefix=crates/sarusctl      sarusctl-upstream      main --squash
git subtree pull --prefix=crates/skybox        skybox-upstream        main --squash

If the subtree pull create new commits (because their main moved forward, you can push them

git push origin main

Then work on a new feature


License

See LICENSE file for this repository. See individual crates for their licensing terms.

About

Rust workspace (“super-crate”) that brings together the main components of the Sarus Suite for the cluster tooling so they can be developed and built from a single checkout.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors