No longer use blob file path to read files but use the zope api instaed. #8218
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
elioschmutz
force-pushed
the
es/TI-3545-fix-blob-access
branch
2 times, most recently
from
fa0d37d to
791d158
Compare
elioschmutz
force-pushed
the
es/TI-3545-fix-blob-access
branch
from
791d158 to
8477694
Compare
This will fix an issue reading non existing files when using a non shared blobs dir: https://relstorage.readthedocs.io/en/latest/relstorage-options.html#blobs In this case, the blobs-directory only aces as a cache. File may be there but not necessarly. If using the zope api, files will be synced as soon as they're needed.
elioschmutz
force-pushed
the
es/TI-3545-fix-blob-access
branch
from
8477694 to
0c192e7
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When running with a non shared blob directory,
RelStorageuses the local filesystem only as a cache for blobs. Once the configuredblob-cache-sizeis reached, cached blobs may be evicted and no longer exist on disk, even though they are still available in ZEO.Several parts of the codebase were reading blob data directly via the committed filesystem path. This approach assumes that the blob file is always present locally, which is not guaranteed in a non shared blob setup. As a result, file access may fail when a blob has been removed from the local cache.
Fix
This change replaces direct filesystem access to blob paths with the official Zope API for reading file data. Files are now read via
document.get_file().read()instead of opening the committed blob path.Using the Zope API ensures that missing blobs are transparently re-synced when needed. This makes file access robust against blob cache eviction. This issue does not exists in the default
FileStorage-setup. In only exists when usingRelStoragein combination withshared-blob-dir=false.See https://relstorage.readthedocs.io/en/latest/relstorage-options.html#blobs for more details about the options.
For TI-3545
Checklist
Everything has to be done/checked. Checked but not present means the author deemed it unnecessary.