Skip to content

Commit 756409f

Browse files
better explanation
1 parent db0070f commit 756409f

File tree

1 file changed

+17
-4
lines changed

1 file changed

+17
-4
lines changed

docs/platforms/mlp/index.md

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -51,16 +51,29 @@ Use scratch to store datasets that will be accessed by jobs, and for job output.
5151
Scratch is per user - each user gets separate scratch path and quota.
5252

5353
* The environment variable `SCRATCH=/iopsstor/scratch/cscs/$USER` is set automatically when you log into the system, and can be used as a shortcut to access scratch.
54+
* There is an additional scratch path mounted on [Capstor][ref-alps-capstor] at `/capstor/scratch/cscs/$USER`.
5455

5556
!!! warning "scratch cleanup policy"
5657
Files that have not been accessed in 30 days are automatically deleted.
5758

5859
**Scratch is not intended for permanent storage**: transfer files back to the capstor project storage after job runs.
5960

60-
!!! note
61-
There is an additional scratch path mounted on [Capstor][ref-alps-capstor] at `/capstor/scratch/cscs/$USER`.
62-
This filesystem should perform better for contiguous reads and writes.
63-
Therefore, we recommend using capstor for storing checkpoint files generated by your training runs.
61+
!!! note "file system suitability"
62+
The Capstor scratch filesystem is based on HDDs and is optimized for large, sequential read and write operations.
63+
We recommend using Capstor for storing **checkpoint files** and other **large, contiguous outputs** generated by your training runs.
64+
In contrast, Iopstor uses high-performance NVMe drives, which excel at handling **IOPS-intensive workloads** involving frequent, random access. This makes it a better choice for storing **training datasets**, especially when accessed randomly during machine learning training.
65+
66+
### Scratch Usage Recommendations
67+
68+
Use Iopstor scratch (`$SCRATCH`) for:
69+
* Training and validation datasets that are read frequently and non-sequentially.
70+
* Workloads that perform many small, random I/O operations.
71+
72+
Use Capstor scratch (`/capstor/scratch/cscs/$USER`) for:
73+
* Storing model checkpoints.
74+
* Outputs from simulations or training jobs that involve large, contiguous I/O.
75+
76+
After your job completes, remember to transfer any important results to your permanent project storage.
6477

6578
### Project
6679

0 commit comments

Comments
 (0)