Option for delayed deletion of files from backing store #538
Description
We are looking at creating an option between delete immediately and delete never - more specifically, we are planning on implementing a 'Delete after X duration' feature.
The idea being, much like the 'never delete' option, that you use the bucket if you restore from a DB backup of a certain age. But we only keep snapshots a certain amount of time, and I'm never a fan of 'never clean up' setups. So we want a retirement strategy for those files, once the related DB snapshots are gone.
There are limitations of this. Namely, just like delete immediately, it is not compatible with multiple sites using the same backing store, and we will need to implement a table that tracks files that have been deleted from mdl_files, but are waiting to be deleted from the backing store. We will then have a cron task that uses that table to go and delete the actual files.
We will also need to check that table when adding a new file to the store, but if a hit is found, it means we won't need to re-upload it - just remove it from the to-be-deleted table.
We already have some developers looking at this, and out plan is to open a PR, but I wanted to open this discussion now, incase there are some glaring problems with this plan that either I'm missing, or would prevent it from being integrated into the community plugin.