You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In our "simple" pytorch example, I counted **22806 inodes**!
69
68
69
+
##### Alternative virtual environment creation using uv
70
+
71
+
The installation process described above is not optimized for file system performance and will still be slow on Lustre filesystems. An alternative way to create the virtual environment is to use the [uv](https://docs.astral.sh/uv/) tool, which supports _relocatable_ virtual environments and asynchronous package downloads for better installation times. This way, the installation process is much shorter and the resulting squashfs image can be shared across projects, as the virtual environment can be safely used from any location.
72
+
73
+
```bash
74
+
# activate the uenv as before
75
+
uenv start prgenv-gnu/24.11:v1 --view=default
76
+
77
+
# create and activate a new relocatable venv using uv
#### Step 2: make a squashfs image of the virtual environment
71
88
72
89
The next step is to create a single squashfs file that contains the whole `$SCRATCH/sqfs-demo/.pyenv` path.
@@ -117,7 +134,7 @@ Note that the original virtual environment is still installed in `$SCRATCH/sqfs-
117
134
A benefit of this approach is that the squashfs file can be copied to a location that is not subject to the Scratch cleaning policy.
118
135
119
136
!!! warning
120
-
Virtual environment are usually not relocatable as they contain symlinks to absolute locations inside the virtual environment. Therefore, you need to mount the image in the exact same location where you created the virtual environment.
137
+
Virtual environments are not relocatable by default as they contain symlinks to absolute locations inside the virtual environment. This means that the squashfs file must be mounted in the exact same location where the virtual environment was created, unless it contains a virtual environment specifically created using a tool with support for relocatable virtual environments (e.g. `uv venv --relocatable` as mentioned in step 1), in which case it can be mounted in any location.
121
138
122
139
#### Step 4: (optional) regenerate the virtual environment
0 commit comments