Marking backups as "in progress" or "finished"

[abbbi]

While working on the sync feature #48 i noticed that for sync the syncjob checks by identifying if an backup is active by using the "Size" flag of an snapshot:

       Ok(snapshot_list
            .into_iter()
            .filter_map(|item: SnapshotListItem| {
                let snapshot = item.backup;
                // in-progress backups can't be synced                                                                                  
                if item.size.is_none() {
                    task_log!(
                        worker,
                        "skipping snapshot {} - in-progress backup",

while we can "fake" this response for the sync to work, the current implementation has no real way to determine if an
backup is still active.

I think we could either use an "active" flag which is added as the backup starts and cleared as it is finished?

[tizbac]

Yes or simply we could keep track of it in the proxy itself, without doing it in S3 since anyway all running backups are necessarily in memory Il mar 20 ago 2024, 10:34 Michael Ablassmeier ***@***.***> ha scritto:

…

While working on the sync feature #48 <#48> i noticed that for sync the syncjob checks by identifying if an backup is active by using the "Size" flag of an snapshot: Ok(snapshot_list .into_iter() .filter_map(|item: SnapshotListItem| { let snapshot = item.backup; // in-progress backups can't be synced if item.size.is_none() { task_log!( worker, "skipping snapshot {} - in-progress backup", while we can "fake" this response for the sync to work, the current implementation has no real way to determine if an backup is still active. I think we could either use an "active" flag which is added as the backup starts and cleared as it is finished? — Reply to this email directly, view it on GitHub <#49>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABBV6SCETSWTQU7IKPX3HLZSL5RLAVCNFSM6AAAAABMZMPF66VHI2DSMVQWIX3LMV43ASLTON2WKOZSGQ3TKMBUGQ3DINQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[tizbac]

Only edge case it comes to my mind is proxy crashing while doing backup and unable to remove it, a clean way without duplication would be setting the corrupt flag and removing it when done

[abbbi]

Only edge case it comes to my mind is proxy crashing while doing backup and unable to remove it, a clean way without duplication would be setting the corrupt flag and removing it when done

yes. Also, signal handling should be added so it could be shutdown "gracefully", means if it receives kill signal and there are still active sessions, wait until they are gone for X seconds, then shutdown.

Maybe it shouldnt write the stuff into the backups/ folder in the first place,but something like active/ first, and if the backup finishes move the directory to the backups folder (atomic).