-
Notifications
You must be signed in to change notification settings - Fork 32
Description
Summary
The client incorrectly emits an error that a token can't be parsed when it was in fact trying to access a namespace with public reads enabled and thus never sent a token in the first place:
ERROR[2026-01-04T23:28:22Z] Failure getting pelican://director:8444/public/data/0.0: failed download from cache-0:8442: Authorization Error: Error code 4000: Permission denied: token could not be parsed
Reproducer
We're going to construct a federation where the cache is horrifically out of date when it comes to understanding what namespaces exist in the federation, let alone their auth policies:
After following the setup instructions in the included README:
-
Visit https://localhost:9000/ and wait for both the origin and cache to report completely healthy. (Filtering on services in an error state will otherwise bite you…)
-
Exec into the dev container:
docker exec -it pelican-dev-1 bash -il. -
Run:
/scratch/bin/pelican object get --debug pelican://director:8444/public/data/0.0 /dev/null.
Details
Pelican version:
Version: 7.22.0
Build Date: 2025-12-19T19:00:13Z
Build Commit: 074a9766f8c06e281c4fbfeb98a205f021288b56
Built By: goreleaser
The problem is that our error classification scheme makes an incorrect assumption:
pelican/client/error_helpers.go
Lines 457 to 465 in c8edb77
| // Handle permission denied errors | |
| var pde *PermissionDeniedError | |
| if errors.As(err, &pde) { | |
| // If the token is expired we can retry because we will just get a new token | |
| // otherwise something is wrong with the token | |
| expired, expiration, tokenErr := tokenIsExpired(tokenContents) | |
| if tokenErr != nil { | |
| pde.message = "Permission denied: token could not be parsed" | |
| pde.expired = false |
In the scenario created above, the director believes that the namespace allows for public reads, and thus the client proceeds without a token. However, the cache doesn't know that the namespace even exists, hence "permission denied."
Fair enough, the client can't know that the cache is behind the times. The client can, however, at least accurately report on whether it tried to send a token in the first place. (Also, one might wonder how the client could make it this far with an unparsable token. Surely, it would have checked it before contacting the cache?)