Skip to content

Add optional TTL-based cleanup for httpcache to reduce disk and memory usage#2218

Open
wswsmao wants to merge 1 commit intocontainerd:mainfrom
wswsmao:fscachettl
Open

Add optional TTL-based cleanup for httpcache to reduce disk and memory usage#2218
wswsmao wants to merge 1 commit intocontainerd:mainfrom
wswsmao:fscachettl

Conversation

@wswsmao
Copy link
Copy Markdown
Contributor

@wswsmao wswsmao commented Jan 8, 2026

Problem

The current disk-backed HTTP chunk cache can consume significant storage over time:

  • Cached chunk files may keep accumulating under the local httpcache directory.
  • Long-running nodes can retain stale cache data much longer than needed.
  • Existing behavior does not provide a built-in expiration mechanism for on-disk HTTP cache entries.

Solution

This PR introduces an optional TTL-based cleanup for disk-backed HTTP chunk cache entries:

  • When http_cache_chunk_ttl_sec > 0 and the HTTP cache uses a disk-backed mode, a background janitor is started during resolver initialization.
  • The janitor periodically scans the httpcache directory and removes expired cache files based on file mtime.
  • The janitor skips the wip directory to avoid interfering with in-progress cache writes.
  • The feature is disabled by default to preserve existing behavior.

Configuration

Add the following config option:

  • http_cache_chunk_ttl_sec
    • Default: 0 (TTL cleanup disabled)
    • > 0: enable TTL cleanup with the given value in seconds (e.g. 120)

Example:

http_cache_chunk_ttl_sec = 120

@wswsmao wswsmao force-pushed the fscachettl branch 2 times, most recently from 4247f3d to 18410c1 Compare January 9, 2026 01:57
@wswsmao
Copy link
Copy Markdown
Contributor Author

wswsmao commented Jan 22, 2026

@ktock

…y usage

Signed-off-by: abushwang <abushwang@tencent.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant