-
Notifications
You must be signed in to change notification settings - Fork 32
Open
Labels
cacheIssue relating to the cache componentIssue relating to the cache componentinternalInternal code improvements, not user-facingInternal code improvements, not user-facing
Milestone
Description
There are a few ongoing issues with Director tests at the cache. First we noticed that unless the cache's high water mark is hit, Director tests aren't cleaned up (sometimes leading to hundreds of thousands of files in a single directory). This in turn starts affecting the filesystem, which bogs down the Cache.
We need two main fixes:
- Director tests should be cleaned up periodically by the Pelican process so we're not relying on XRootD's purge process. This unfortunately means we need to deal with these files being owned by XRootD, but being modifiable by Pelican (a pitfall with
dropprivs). - Director test files are already uniquely timestamped, but we need to nest them in a daily directory. That is, every day we should use a new directory to make sure that no directory contains more than a single day's worth of files.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
cacheIssue relating to the cache componentIssue relating to the cache componentinternalInternal code improvements, not user-facingInternal code improvements, not user-facing