Tombstone Implementation Notes changes

[zimeon]

From use case Flagging File Corruption #14, Supporting Physical File Level Deletion #42, and Collapsing Object Versions #46. See also #669 for spec changes.

[zimeon]

Summary of proposed items to cover in implementation notes based on 2025-08-18 Extended Editors Meeting

Why might a file be tombstoned?

• Missing/vanished, removed/deleted -- spec will not distinguish these cases: file does not appear in manifest, appears with original digest in tombstone. Message provides a way to record this for humans.
• Corrupted, but readable -- file appears with “new” digest in manifest, with original digest in tombstones
• Present or listed in file system but unreadable or not reliably readable -- file appears with empty digest string (not a valid digest output in any digest format we know, and an empty string is valid as key on JSON object, whereas Null etc. are not) in manifest, with original digest in tombstones

Why might you not tombstone?

• Because you really want to purge the file and all record of it (this is a hard delete, a thanos snap)
• Because you're okay with soft deletion (file is still in object, but not in current version state, aka "OCFL Classic Delete Flavor", it's a logical unlinking)

Use cases for using tombstone for corrupted and or unreadable:

• write once storage where we can’t delete
• corruption, where we want to keep the corrupted file for possible later analysis

What if a file later "comes back" or is found?

• Add back with new content path (in the new version content directory)
• Old version states become unpackable again if new content used with the digest to "refer forward" to the new content path

Implementation notes must:

• Account for deduped files
• Talk about read errors and inconsistency
• Talk about the corruption characteristics of different storage types
• Talk about the need for documentation in the new version
• Connection with other OCFL v2 features - packages (if a package were corrupt then many tombstones may be added) and content-linking
• Validation strategies
  • Validation rules for an unreadable/unreliable readable file, one that is marked with a blank digest in the manifest:
    • Check for presence in storage
    • Do not check content
• It is necessary to read head inventory.json before reading/validating any prior version. This is the usual approach anyway (and prior version inventories are optional in OCFL v1) but with OCFL v2 there is additional need to understand any tombstones before attempting validation of prior versions.