Skip to content

Bug: Memory leak and performance issue in purge() using KEYS command #5

@tanbro

Description

@tanbro

Severity

High

Location

File: src/redis_func_cache/policies/base.py
Lines: 188-217

Description

In BaseMultiplePolicy.purge() and apurge(), the method uses KEYS pattern matching with wildcards:

pattern = f"{self.cache.prefix}{self.cache.name}:{self.__key__}:*"
keys = self.client.keys(pattern)

Problems:

  1. O(N) operation: KEYS blocks the Redis server while scanning the entire keyspace
  2. Too broad matching: Pattern could match unintended keys if prefix/name contain special characters
  3. No validation: No check that matched keys actually belong to this cache instance

Impact

  • Potential data loss from accidentally deleting unrelated cache keys
  • Performance degradation on large Redis databases
  • Redis blocked during KEYS execution affecting other operations

Reproduction Steps

# Create cache with similar name
cache1 = RedisFuncCache("my-cache", LruTMultiplePolicy(), ...)
cache2 = RedisFuncCache("my-cache-extra", LruTMultiplePolicy(), ...)

cache1.purge()  # May delete keys from cache2

Proposed Solution

  1. Use Redis SCAN instead of KEYS:
def purge(self) -> int:
    pattern = f"{self.cache.prefix}{self.cache.name}:{self.__key__}:*"
    keys = []
    for key in self.client.scan_iter(match=pattern, count=100):
        keys.append(key)
    if keys:
        return self.client.delete(*keys)
    return 0
  1. Add more specific pattern matching with hash tags
  2. Validate prefix/name characters during initialization

Additional Context

  • Redis documentation warns against KEYS in production
  • SCAN is non-blocking but returns keys in cursor-based pagination
  • Affects all Multiple*Policy variants

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions