Added code to make st-archive handle DART files

[kdraeder]

Description of changes

Make st_archive handle compressed (.gz) CLM output files.

Specific notes

This is very helpful for (large multi-instance, multiple short hindcast) data assimilation experiments.
config_archive.xml already handles DART's .e. files.

Contributors other than yourself, if any: none

CTSM Issues Fixed (include github issue #): #3082

Are answers expected to change (and if so in what way)? No

Any User Interface Changes (namelist or namelist defaults changes)? No

Does this create a need to change or add documentation? Did you do so? I don't believe so at this time.
Later it may be useful to add documentation about how DART files are archived, to help users find them.

Testing performed, if any:
I ran a B compset built from the tags in .gitmodules, but no development branches checked out.
I ran st_archive on the output to confirm that it handled all files correctly.
I checked out a feature branch based on master, added the code changes, and pushed to the fork
on kdraeder.
See CIME PR #4788 for details.

[wwieder]

Erik will bring this to b4b-dev

[ekluzek]

We should rebase this to b4b-dev and bring it in there.

[ekluzek]

We weren't bringing this in because of other issues with rpointer files. But, those are resolved now, so this should come in.

I'm rebasing to b4b-dev.

[ekluzek]

@kdraeder thanks for sending this in. We had neglected this for a while, because we had issues with rpointer files to be resolved, and I wanted that to be resolved first. But, now is a good time to bring it in.

Can I ask about the need to optionally include the gzipped history files? Is this something DART does because there are so many instances? And does it then ungzip them -- or how does that part happen?

[kdraeder]

Yes, we often need to compress files because there a lots of instances and because we do lots of short forecasts, so disk space gets used in a hurry. I realize that assuming any compression will be gzip is not robust, and it clutters the code somewhat. There may be better solutions, like, does the pattern need to have the extension specified? Or would we leave off the .nc and .gz? Thanks for pushing this along when it makes sense to. Kevin

…

On Tue, Oct 7, 2025 at 11:46 AM Erik Kluzek ***@***.***> wrote: ***@***.**** approved this pull request. @kdraeder <https://github.com/kdraeder> thanks for sending this in. We had neglected this for a while, because we had issues with rpointer files to be resolved, and I wanted that to be resolved first. But, now is a good time to bring it in. Can I ask about the need to optionally include the gzipped history files? Is this something DART does because there are so many instances? And does it then ungzip them -- or how does that part happen? ------------------------------ In cime_config/config_archive.xml <#3084 (comment)>: > @@ -2,9 +2,9 @@ <comp_archive_spec compname="clm" compclass="lnd"> <rest_file_extension>r</rest_file_extension> <rest_file_extension>rh\d?</rest_file_extension> - <hist_file_extension>h\d*.*\.nc$</hist_file_extension> - <hist_file_extension>lilac_hi.*\.nc$</hist_file_extension> - <hist_file_extension>lilac_atm_driver_h\d*.*\.nc$</hist_file_extension> + <hist_file_extension>h\d*.*\.nc(\.gz)?$</hist_file_extension> ⬇️ Suggested change - <hist_file_extension>h\d*.*\.nc(\.gz)?$</hist_file_extension> + <!-- History files including those for LILAC, optionally include gzipped files which is needed for DART --> + <hist_file_extension>h\d*.*\.nc(\.gz)?$</hist_file_extension> — Reply to this email directly, view it on GitHub <#3084 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADFFGEC5KCTI2ANGOJVEYXT3WP37XAVCNFSM6AAAAAB3XMF7A6VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZTGMJRGIZTONJVGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[ekluzek]

@kdraeder could you add me as a collaborator to your CTSM fork?

[ekluzek]

Ensured that IRT_Ld11.f10_f10_mg37.IHistClm60BgcCrop.derecho_gnu.clm-default ran and compare identical to baseline. Merging.

[kdraeder]

@ekluzek Thanks for pushing this through before the CESM3 release!