-
Notifications
You must be signed in to change notification settings - Fork 10
Open
Labels
semver: new featureNew feature (will increment minor version)New feature (will increment minor version)
Description
What happened?
Based on @wlin7 report:
When an archiving is stopped prematurely, zstash can resume with the updatefunction, but the zstash update can spend many hours (even days depending on simulation size) just to gather the files that have not been archived in previous runs of zstashfor the same simulation before resuming actual archiving. We need to dramatically cut down the time for this step by improving the inner working of zstash update, or adjusting the on-disk file management.
What machine were you running on?
chrysalis
Environment
latest in e3sm unified 1.12
Minimal Complete Verifiable Example (MCVE)
Relevant log output
Anything else we need to know?
@golaz point out changes need to be made at
Line 74 in 01ffe34
| def get_files_to_archive(cache: str, include: str, exclude: str) -> List[str]: |
Metadata
Metadata
Assignees
Labels
semver: new featureNew feature (will increment minor version)New feature (will increment minor version)