Skip to content

Storage: Add warning for mounting location #38

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 12, 2025
Merged

Storage: Add warning for mounting location #38

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
01d1f07
Add warning for mounting location
havogt Mar 12, 2025
ae7f2f5
Update docs/guides/storage.md
bcumming Mar 12, 2025
5837e15
Update docs/guides/storage.md
bcumming Mar 12, 2025
e77c45c
Update docs/guides/storage.md
bcumming Mar 12, 2025
bd04199
Merge branch 'main' into patch-1
bcumming Mar 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions docs/guides/storage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ The main reasons for this include:
As a result, using virtual environments can be slow, and these problems are only exacerbated when the virtual environment is loaded simultaneously by many ranks in an MPI job.

One solution is to use the tool `mksquashfs` to compresses the contents of a directory - files, inodes and sub-directories - into a single file.
This file can be mounted as a read-only file [Squashfs](https://en.wikipedia.org/wiki/SquashFS) file system, which is much faster because a single file is accessed instead of the many small files that were in the original environment.
This file can be mounted as a read-only [Squashfs](https://en.wikipedia.org/wiki/SquashFS) file system, which is much faster because a single file is accessed instead of the many small files that were in the original environment.


#### Step 1: create the virtual environment
Expand Down Expand Up @@ -95,7 +95,7 @@ mksquashfs $SCRATCH/sqfs-demo/.pyenv pyenv.squashfs \
The default installed version of `mksquashfs` on Alps does not support the best `zstd` compression method.
Every uenv contains a better version of `mksquashfs`, which is used by the uenv to compress itself when it is built.

The exact location inside the uenv depends on the target architecure, and version, and will be of the form:
The exact location inside the uenv depends on the target architecture, and version, and will be of the form:
```
/user-environment/linux-sles15-${arch}/gcc-7.5.0/squashfs-${version}-${hash}/bin/mksquashfs
```
Expand All @@ -116,6 +116,9 @@ Note that the original virtual environment is still installed in `$SCRATCH/sqfs-

A benefit of this approach is that the squashfs file can be copied to a location that is not subject to the Scratch cleaning policy.

!!! warning
Virtual environment are usually not relocatable as they contain symlinks to absolute locations inside the virtual environment. Therefore, you need to mount the image in the exact same location where you created the virtual environment.

#### Step 4: (optional) regenerate the virtual environment

The squashfs file is immutable - it is not possible to modify the contents of `.pyenv` while it is mounted.
Expand Down