Append ifgramStack.h5 with new pairs

[tshreve]

ASF's multi-burst feature now allows for consistent spatial coverage of up to 15 Sentinel-1 bursts. When processing 6+ bursts and approx. 900 interferograms spanning 2015-2025, the 'load_data' step in MintPy can take up to 3 hours on my workstation. Currently, I am rerunning this step each time I want to update the time series. I was thinking it would be easier to append the ifgramStack.h5 file with new pairs as images are acquired, as opposed to recreating this file each time.

Describe the solution you'd like
I'm interested in adding a line to the config file 'load_data' step, which creates a function to check if pair dates exist in ifgramStack.h5. It also checks if the file size is consistent. If the date exists and file size is the same, then 'load_data' only writes non-existing pairs to ifgramStack.h5.

Describe alternatives you have considered
I was wondering if a similar functionality or work-around already exists.

Are you willing to help implement and maintain this feature?

• Yes
• No

[welcome]

👋 Thanks for opening your first issue here! Please filled out the template with as much details as possible. We appreciate that you took the time to contribute!
Make sure you read our contributing guidelines.

[scottstanie]

You could look into switching on the resizable dataset option option to layout_hdf5 (or some other function that creates the hdf5)

[tshreve]

I was able to get a version of this feature working, but also realized that I was accessing from/saving data to my RAID HDDs, not my workstation's SSD. Working on the SSD directly sped up the interferogram loading ~20x, making this feature not as necessary.

If others are still interested in this feature, let me know, and I can submit a PR.