Module for filtering exports#80
Merged
bbockelm merged 4 commits intoPelicanPlatform:mainfrom Mar 16, 2025
Merged
Conversation
The new plugin, `libXrdOssFilter.so`, allows the administrator to only permit the opening / listings of files and directories that match specified globs or prefixes. This can be used to keep some storage from being visible via XRootD (even with the appropriate permissions). For example, to hide all files and directories beginning with "." from being exported, one could setup: ``` ofs.osslib ++ libXrdOssFilter.so filter.glob /** ``` (This policy would be quite difficult to implement via the existing authorization frameworks)
The `filter.prefix` approach is meant to provide a simpler variant of `filter.glob` (for administrators who feel they may get tripped up). To prevent unexpected behaviors, ensure that there are no glob metacharacters in the provided path and that the path is normalized.
Collaborator
|
Hi @bbockelm ! I'm still digesting this PR since it's built on top of stuff I don't understand yet. I'll leave a comment or two but you should proceed when you're ready. |
Collaborator
Author
|
Ok, going to take PI privilege here and move this forward. This isn't in active use yet and has unit test coverage. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR (note: it's built on top of #79 and hence it contains those commits -- ignore them for review purposes or wait until that PR is merged) adds a new module,
libXrdOssFilter, for managing what objects are exported via XRootD.libXrdOssFiltercan be stacked on top of the HTTP or S3 OSS plugin and provides reasonably fine-grained control over what data is accessible. The administrator can limit things to specific prefixes or only objects matching a certain glob (e.g.,/ncar/dataset*/**or/foo/*.png). It supports the "globstar" operator, allowing multiple path hierarchies to be matched (e.g.,/pelican/**/*.gowill export all*.gofiles underneath/pelican). The integration test contains a valid XRootD configuration file demonstrating this.Includes reasonable unit test and a simple end-to-end integration test built on top of the S3 integration test.