Skip to content

Conversation

@bcumming
Copy link
Member

No description provided.

@github-actions
Copy link

preview available: https://docs.tds.cscs.ch/240

Copy link
Member

@RMeli RMeli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the "Recommendations on run configuration" section stick around somewhere in the documentation as general recommendations for large scale runs?

@github-actions
Copy link

preview available: https://docs.tds.cscs.ch/240

Copy link
Collaborator

@msimberg msimberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for rebasing this @bcumming. I'll look into integrating some of the tips and tricks into other sections.

@msimberg msimberg added this pull request to the merge queue Sep 24, 2025
Merged via the queue into eth-cscs:main with commit e8165c9 Sep 24, 2025
3 checks passed
github-merge-queue bot pushed a commit that referenced this pull request Nov 7, 2025
This adds the `lfs migrate` recommendation/example that was removed
along with the Gordon-Bell guide (#240).

This also adds an example to recursively update all files in a directory
(finding files using `lfs find`).

@mpasserini can you or someone else from the storage team comment if you
think these are ok recommendations. For example:
- (this already exists) do you think the recommended block sizes and
progressive layouts are sane recommendations?
- Are there any concerns with using e.g. `xargs -P` to migrate files in
parallel (the example is currently sequential which can be slow, but I'm
not sure if doing a parallel `lfs migrate` adds unnecessarily high load
on lustre)?

Note that I changed the short options (`-c`/`-S`/`-E`) to long options
as I find that it's easier to follow for someone like me who doesn't
know the options by heart. However, the commands do end up being longer
(wider than the regular page, and require scrolling), and I'm happy to
change that back (it's not the important part of this PR).

I'm not attempting to move or restructure the storage guide otherwise.

Link to generated docs:
https://docs.tds.cscs.ch/297/guides/storage/#lustre-tuning
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants