Inconsistent usage of set method in custom cache handler for shared cache management on Amazon EFS

[LorenzoGalassi]

Summary

Hello everyone,
I am trying to make a custom cache handler because I have an application on ECS and I would like to use a shared cache by having it reside on a shared file system (EFS).

I've currently gotten this far with the code:

// eslint-disable-next-line @typescript-eslint/no-var-requires
const fs = require("fs/promises");
// eslint-disable-next-line @typescript-eslint/no-var-requires
const path = require("path");

const isLocalDevelopment = process.env.LOCAL_DEV_CACHE === "true";

const CACHE_DIR = isLocalDevelopment
    ? "./my-path"
    : "/efs/my-path";

module.exports = class CacheHandler {
    /**
     * Initializes the cache handler.
     * @param {Object} options - Any options you might need.
     */
    constructor(options) {
        this.options = options;
        // Ensure that the cache directory exists; create it if it doesn't.
        fs.mkdir(CACHE_DIR, { recursive: true })
            .then(() =>
                console.log(
                    `[CACHE_HANDLER] | Cache directory ready at ${CACHE_DIR}`,
                ),
            )
            .catch(err =>
                console.error(
                    "[CACHE_HANDLER] | Error creating cache directory:",
                    err,
                ),
            );
    }

    /**
     *
     * @param {string} key
     */
    _formatKey(key) {
        return `${key.slice(0, 3)}...${key.slice(-3)}`;
    }

    /**
     * Constructs a safe file path for a given cache key.
     * @param {string} key - The cache key.
     * @returns {string} - The full path to the cache file.
     */
    _getFilePath(key) {
        // Encode the key to avoid issues with special characters.
        return path.join(CACHE_DIR, encodeURIComponent(key) + ".json");
    }

    /**
     * Retrieves a value from the persistent cache.
     * @param {string} key - The cache key.
     * @returns {Promise<any>} - The cached value or null if not found.
     */
    async get(key) {
        const filePath = this._getFilePath(key);
        try {
            const data = await fs.readFile(filePath, "utf8");
            const cacheEntry = JSON.parse(data);
            console.log(
                `[CACHE_HANDLER] | Cache hit for key "${this._formatKey(key)}".`,
            );
            return cacheEntry.value;
        } catch (err) {
            // Log the cache miss or error, and return null.
            console.warn(
                `[CACHE_HANDLER] | Cache miss or error reading key "${this._formatKey(key)}":`,
                err,
            );
            return null;
        }
    }

    /**
     * Stores a value in the persistent cache.
     * @param {string} key - The cache key.
     * @param {any} data - The data to cache.
     * @param {Object} ctx - Additional context such as tags for invalidation.
     */
    async set(key, data, ctx) {
        const filePath = this._getFilePath(key);
        const cacheEntry = {
            value: data, // The cached data.
            lastModified: Date.now(), // Timestamp for this cache entry.
            tags: ctx.tags || [], // Tags for invalidation purposes.
        };

        try {
            // Write the cache entry to the file as JSON.
            await fs.writeFile(filePath, JSON.stringify(cacheEntry), "utf8");
            console.log(
                `[CACHE_HANDLER] | Cache entry set for key "${this._formatKey(key)}"`,
            );
        } catch (err) {
            console.error(
                `[CACHE_HANDLER] | Error writing cache for key "${this._formatKey(key)}":`,
                err,
            );
        }
    }

    /**
     * Invalidates cache entries based on one or more tags.
     * @param {string|string[]} tags - The tag(s) to use for invalidation.
     */
    async revalidateTag(tags) {
        // Normalize the tags to an array.
        tags = [tags].flat();

        try {
            // Read all files in the cache directory.
            const files = await fs.readdir(CACHE_DIR);
            for (const file of files) {
                const filePath = path.join(CACHE_DIR, file);
                try {
                    const data = await fs.readFile(filePath, "utf8");
                    const cacheEntry = JSON.parse(data);
                    // If the cache entry contains any of the invalidation tags, delete the file.
                    if (cacheEntry.tags.some(tag => tags.includes(tag))) {
                        await fs.unlink(filePath);
                        console.log(
                            `[CACHE_HANDLER] | Invalidated cache file "${file}" due to tag match.`,
                        );
                    }
                } catch (err) {
                    console.error(
                        `[CACHE_HANDLER] | Error processing cache file "${file}":`,
                        err,
                    );
                }
            }
        } catch (err) {
            console.error(
                " [CACHE_HANDLER] | Error reading cache directory during revalidation:",
                err,
            );
        }
    }
};

The problem I am experiencing is that there are “SET” logs after cache hits without a revalidation between the two events. The logic I expect to happen internally in the framework is:

1. Fetch
2. GET on the cache
   3.1 If GET = null => I fetch and use a SET on the cache
   3.2 If GET != null => I do not perform the fetch

The only case where GET = null is if the call was never cached or if a revalidation was invoked, both of which are not happening in my local tests after a cache hit that I detecto from logs.

I ask: am I missing something?

N.B. obviously there are no revalidate = 0 or similar in the code, if I remove the custom cache handler the caching works correctly

Additional information

No response

Example

No response

[icyJoseph]

Just to see if I am understanding, from your side, if the getting data, gives null, you fetch and set, otherwise, no work is done.

And it is not possible for the stored value, to ever be null? Can you inspect the ECS fs, to see what's getting stored? Or if you have the permissions needed?

I feel like I've seen a somewhat similar issue somewhere.

Also, local production builds work fine too right?

[icyJoseph]

Ok, not, it looked similar, but it was unrelated: #75044 (comment)

[LorenzoGalassi]

@icyJoseph The problem is that we do not control the usage flow of SET and GET, it is handled by the Next.js internals when using fetch. What we notice is that unnecessary SETs are occurring, immediately after a HIT cache a SET of the same fetch is re-performed, even though a revalidate did not occur in the meantime.