rethink old metadata file deletion

[jku]

Currently previous versions of targets are deleted as the new one is created. This creates a situation where the repo is invalid until snapshot is updated.

Try to leave the old files in place, and only remove them during the snapshot update. This way the target update can be merged and the repo is valid while snapshot is getting updated.

tufrepo verify must be able to handle this and should recognise the issue ("the repo is valid but there are metadata files that are not included in snapshot")

[jku]

I think verify should be able to recognize at least these situations:

• valid root, valid but expired timestamp: can be fixed with timestamp update
• valid root, valid timestamp, expired snapshot: can be fixed with snapshot update
• a metadata file is part of delegation tree, but not part of snapshot (either completely new role or version is > version in snapshot): updating snapshot should work. Should make sure metadata version is reasonable though: can be fixed with snapshot update
• valid but expired (delegated) targets file. This can't be handled with TrustedMetadataSet currently: warn, but no error
• ERROR: metadata file exists that is not part of delegation tree AND not part of snapshot (old metadata is allowed to be part of snapshot but not part of delegation tree): unrecoverable situation
• ERROR: loading the delegation tree fails somehow: unrecoverable situation
• delegation tree is valid and all metadata is part of the snapshot and is in delegation tree. Note that snapshot may contain metadata that does not exist : no action needed

theupdateframework/python-tuf#1524 will allow handling all cases I believe, except handling expired targets files.