CacheableMemory leaks memory and throws errors when hashCache faces the limit

[PaulAnnekov]

Describe the bug

CacheableMemory has an internal hash key cache (it's a Map). Every time you try to set/get some value by key, the key is first checked in hash key and then hashed + added if not found. I couldn't find code which manages the size of the Map. If your key set is not static (and usually, it's not), this Map grows infinitely, consuming more and more memory, until you will face with Map size limit (17M entries). Here is the place where error is thrown at the end:

cacheable/packages/cacheable/src/memory.ts

Line 480 in ca02913

this._hashCache.set(key, result);

And here is an example of the error:

RangeError: Map maximum size exceeded\n    at Map.set (<anonymous>)\n    at CacheableMemory.hashKey (/home/nodejs/app/node_modules/cacheable/dist/index.cjs:709:21)\n    at CacheableMemory.getStore (/home/nodejs/app/node_modules/cacheable/dist/index.cjs:651:24)\n    at CacheableMemory.get (/home/nodejs/app/node_modules/cacheable/dist/index.cjs:444:24)\n    at Keyv.get (/home/nodejs/app/node_modules/keyv/dist/index.cjs:478:33)\n    at Cacheable.get (/home/nodejs/app/node_modules/cacheable/dist/index.cjs:1378:36)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

How To Reproduce (best to provide workable code or tests!)

Call CacheableMemory.get() > 17M times with different values.

[jaredwray]

@PaulAnnekov - thanks so much for sending this over and yes CacheableMemory does have a 17M entry limit at this time because of the hash map. In the future we are looking to make this even more scalable with Array. This should be out by end of year.

To get around this error you can set the cache to 15M or so with the lru option so it doesnt error out for you. Here are the options for CacheableMemory https://cacheable.org/docs/cacheable/#cacheablememory---in-memory-cache

Here is a code example of how to do this:

import {Cacheable, CacheableMemory} from 'cacheable';

const options = {
  lruSize: 15000000,
}
const primary = new CacheableMemory(options);

const cache = new Cacheable({primary});

This would make sure that you are not hitting that limit issue anymore as we are setting the size limit to 15M.

[PaulAnnekov]

@jaredwray looks like you missed the main point. lruSize won't help, as it controls the cache size, not hash cache size. It's different things.

The following code will cause memory leak and will either die by OOM (if you have not enough free memory) or throw an error. That's despite the small 1000 cache entries limit:

import { Cacheable } from 'cacheable'
import Keyv from 'keyv'

const options = {
  namespace: 'memory',
}
const primary = new Keyv({ ...options })

const cache = new Cacheable({ primary })

async function main() {
  for (let i = 0; i < 18000000; i++) {
    await cache.set(`account:${i}`, i)
    console.log(`account:${i}`, i)
  }
}

main()

[jaredwray]

@PaulAnnekov - interesting. I am working on a solution to make this a bit better and more performant also where you can specify the amount of hash stores you have and also get rid of the _hashCache so you can go past the 17mm limit. Note that most modules have this issue as they use Map such as quick-lru, lru.min, etc...

In this new implementation if you do use the lruSize you will be limite to the Map maximum as we use a DoubleLinkedList to make that work correctly.

Should have this live in the next 30 days or so.

#1126