Skip to content

Clean up Director tests at Cache #3065

@jhiemstrawisc

Description

@jhiemstrawisc

There are a few ongoing issues with Director tests at the cache. First we noticed that unless the cache's high water mark is hit, Director tests aren't cleaned up (sometimes leading to hundreds of thousands of files in a single directory). This in turn starts affecting the filesystem, which bogs down the Cache.

We need two main fixes:

  1. Director tests should be cleaned up periodically by the Pelican process so we're not relying on XRootD's purge process. This unfortunately means we need to deal with these files being owned by XRootD, but being modifiable by Pelican (a pitfall with dropprivs).
  2. Director test files are already uniquely timestamped, but we need to nest them in a daily directory. That is, every day we should use a new directory to make sure that no directory contains more than a single day's worth of files.

Metadata

Metadata

Assignees

Labels

cacheIssue relating to the cache componentinternalInternal code improvements, not user-facing

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions