Skip to content

Update CE documentation for SquashFS mount #156

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 17, 2025
Merged

Update CE documentation for SquashFS mount #156

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion docs/software/container-engine/edf.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ If false, the container filesystem is read-only.
| **Type** | array |
| **Default** | `[]` |

List of bind mounts in the format `SOURCE:DESTINATION[:FLAGS]`. Flags are optional and can include `ro`, `private`, etc.
List of mounts in the format `SOURCE:DESTINATION[:FLAGS]`. By default, it performs bind mount unless the only flag is `sqsh`, in which it performs [a SquashFS mount][ref-ce-run-mounting-squashfs]. When bind mounting, the flags are forwarded to the system mount operation (e.g., `ro` or `private`).

!!! example
* Literal fixed mount map
Expand All @@ -145,6 +145,11 @@ List of bind mounts in the format `SOURCE:DESTINATION[:FLAGS]`. Flags are option
mounts = ["${SCRATCH}:/scratch"]
```

* Mounting a SquashFS image `${SCRATCH}/data.sqsh` to `/data`
```toml
mounts = ["${SCRATCH}/data.sqsh:/data:sqsh"]
```

!!! note
* Mount flags are separated with a plus symbol, for example: `ro+private`.

Expand Down
44 changes: 44 additions & 0 deletions docs/software/container-engine/run.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -182,3 +182,47 @@ Using the `enroot import` documentation page as a reference:
machine registry.ethz.ch login <username> password <GITLAB_TOKEN>
machine gitlab.ethz.ch login <username> password <GITLAB_TOKEN>
```

## Working with storage

Directories outside a container can be *mounted* inside a container so that the job inside the container can read/write on them. The directories to mount should be specified in EDF with `mounts`.

!!! example "Specifying directories to mount in EDF"
* Mount `${SCRATCH}` to `/scratch` inside the container

```toml
mounts = ["${SCRATCH}:/scratch"]
```

* Mount `${SCRATCH}` to `${SCRATCH}` inside the container

```toml
mounts = ["${SCRATCH}:${SCRATCH}"]
```

* Mount `${SCRATCH}` to `${SCRATCH}` and `${HOME}/data` to `${HOME}/data`

```toml
mounts = ["${SCRATCH}:${SCRATCH}", "${HOME}/data:${HOME}/data"]
```

!!! note
The source (before `:`) should be present on the cluster: the destination (after `:`) doesn't have to be inside the container.

See [the EDF reference][ref-ce-edf-reference] for the full specifiction of the `mounts` EDF entry.


[](){#ref-ce-run-mounting-squashfs}
### Mounting a SquashFS image

!!! warning
This feature is only available on some vClusters (Daint and Santis, as of 17.06.2025).

A SquashFS image, essentially being a compressed data archive, can also be mounted _as a directory_ so that the image contents are readable inside the container. For this, `:sqsh` should be appended after the destination.

!!! example "Mounting a SquashFS image `${SCRATCH}/data.sqsh` to `/data`"
```toml
mounts = ["${SCRATCH}/data.sqsh:/data:sqsh"]
```

This is particularly useful if a job should read _multiple_ data files _frequently_, which may cause severe file access overheads. Instead, it is recommended to pack data files into one data SquashFS image and mount it inside a container. See the *"magic phase"* in [this documentation](https://tldp.org/HOWTO/SquashFS-HOWTO/creatingandusing.html) for creating a SquashFS image.