You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Lustre uses *metadata* servers to store and query metadata which is basically what is shown by `ls`: directory structure, file permission, modification dates,...
122
+
As shown in the schema above, Lustre uses *metadata* servers to store and query metadata, which is basically what is shown by `ls`: directory structure, file permission, and modification dates.
125
123
Its performance is roughly the same on [Capstor][ref-alps-capstor] and [Iopsstor][ref-alps-iopsstor].
126
-
This data is globally synchronized, which means that handling many small files is not especially suited for Lustre, see the discussion on [how to handle many small files][ref-guides-storage-small-files].
124
+
This data is globally synchronized, which means Lustre is not well suited to handling many small files, see the discussion on [how to handle many small files][ref-guides-storage-small-files].
127
125
128
126
The data itself is subdivided in blocks of size `<blocksize>` and is stored by Object Storage Servers (OSS) in one or more Object Storage Targets (OST).
129
-
The blocksize and number of OSTs to use is defined by the striping settings.
130
-
A new file or directory ihnerits them from its parent directory.
131
-
The `lfs getstripe <path>` command can be used to get information on the actual stripe settings.
132
-
For directories and empty files `lfs setstripe --stripe-count <count> --stripe-size <size> <directory/file>` can be used to set the layout. The simplest way to have the correct layout is to copy to a directory with the correct layout
127
+
The blocksize and number of OSTs to use is defined by the striping settings, which are applied to a path, with new files and directories ihneriting them from their parent directory.
128
+
The `lfs getstripe <path>` command can be used to get information on the stripe settings of a path.
129
+
For directories and empty files `lfs setstripe --stripe-count <count> --stripe-size <size> <directory/file>` can be used to set the layout.
130
+
The simplest way to have the correct layout is to copy to a directory with the correct layout
131
+
132
+
!!! tip "A blocksize of 4MB gives good throughput, without being overly big..."
133
+
... so it is a good choice when reading a file sequentially or in large chunks, but if one reads shorter chunks in random order it might be better to reduce the size, the performance will be smaller, but the performance of your application might actually increase.
134
+
See the [Lustre documentation](https://doc.lustre.org/lustre_manual.xhtml#managingstripingfreespace) for more information.
133
135
134
-
A blocksize of 4MB gives good throughput, without being overly big, so it is a good choice when reading a file sequentially or in large chunks, but if one reads shorter chunks in random order it might be better to reduce the size, the performance will be smaller, but the performance of your application might actually increase.
@@ -149,7 +149,8 @@ With it it is possible to create a Progressive file layout switching `--stripe-c
149
149
### Iopsstor vs Capstor
150
150
151
151
[Iopsstor][ref-alps-iopsstor] uses SSD as OST, thus random access is quick, and the performance of the single OST is high.
152
-
[Capstor][ref-alps-capstor] on another hand uses harddisks, it has a larger capacity, and it also have many more OSS, thus the total bandwidth is larger. See for example the [ML filesystem suitability][ref-mlp-storage-suitability].
152
+
[Capstor][ref-alps-capstor] on another hand uses harddisks, it has a larger capacity, and it also have many more OSS, thus the total bandwidth is larger.
153
+
See for example the [ML filesystem guide][ref-mlp-storage-suitability].
0 commit comments