astro:cache core component

[FineWolf]

Summary

Create a cache interface within Astro to simplify the development of integrations requiring binary blobs / JSON value caching, and of user space components requiring caching of binary blobs / JSON value.

Background & Motivation

At the moment, in user space, there is no way of effectively caching content that is costly to generate. One common use-case for this is the creation of Astro Endpoints to generate OpenGraph share images.

Integrations are also encouraged to use config.cacheDir to cache results of expensive operations / transformation, but there is no guidance or standard way of doing so.

astro:cache would be a new core component available to both integrations and user space components. This integration would provide by default two adapters:

• NodeFsCacheAdapter: the default configured provider that allows interaction with config.cacheDir. Used at build time, and in user space for static targets.
• InMemoryCacheAdapter: Cache provider that stores values in-memory. Compatible with all targets/runtime.

Third-party integrations would be able to provide and expose their own cache adapters (for example: AWS S3 Cache Adapter, Redis Cache Adapter, etc.).

Possible Use Cases

• astro:assets – uniforming the caching mechanism for astro:assets to astro:cache.
• User space endpoints generating images (for sharing or other purposes) via Satori or canvaskit-wasm could now cache their results for faster build times.

Goals

• Providing a standard interface to read and set binary BLOBs and JSON values into the cache.
• Allow interacting with the cache in user space without requiring the development of a custom integration to do so.
• Normalizing and codifying the use of the cache folder in order to prevent collisions between integrations / user space.

Example

astro:cache would export the following for use in user space and in integrations:

declare module "astro:cache" {
  import type { ZodSchema } from "zod";

  /**
   * Returns a scoped BinaryCacheProvider
   * @param namespace Namespace with which to scope the returned cache provider
   */
  export function getCache(namespace: string): BinaryCacheProvider;

  /**
   * Returns a scoped CacheProvider
   * @param namespace Namespace with which to scope the returned cache provider
   * @param schema Schema which which to optionally validate the unserialized values with
   */
  export function getValueCache<T>(
    namespace: string,
    schema?: ZodSchema<T> | undefined,
  ): CacheProvider<T>;

  /**
   * Helper function to generate a cache key from a given serializable value (for example, a ImageMetadata object).
   * Usage of this is completely optional. An example implementation would be a SHA-256-SUM of the serialized JSON value.
   *
   * @param sourceObj Any JSON serializable object
   */
  export function generateCacheKey<T>(
    sourceObj: T & Serializable<T>,
    options?: GenerateCacheKeyOptions | undefined,
  ): Promise<string>;

  interface GenerateCacheKeyOptions {
    prefix?: string;
    suffix?: string;
  }

  /**
   * A cache provider is a facade of a cache adapter within a given namespace in order to provide
   * user space caching of data.
   */
  export interface CacheProvider<T extends Uint8Array | (T & Serializable<T>)> {
    /**
     * Constructs a new cache provider; This is handled by the factory methods above.
     * @param namespace Namespace of this cache provider. (PRIVATE)
     * @param cacheAdapter Cache adapter associated with this namespace. (PRIVATE)
     */
    constructor(namespace: string, cacheAdapter: CacheAdapter);

    /**
     * Retrieves a value from cache.
     * @param key Cache key
     * @returns {T} The cached data
     */
    get(key: string): Promise<T>;

    /**
     * Retrieve a value from cache. If not present, the value factory is called, and the result of the factory is cached automatically.

     * @param key Cache key
     * @param valueFactory Function that creates a value to be cached. Called when fetching the value from cache failed.
     * @returns {T} The cached data or the newly created data
     */
    getOrCreate(key: string, valueFactory: () => Promise<T> | T): Promise<T>;

    /**
     * Returns true if an item exists in cache, false if not.
     * @param key Cache key
     * @returns {boolean} True if an item exists in cache, false if not.
     */
    has(key: string): Promise<boolean>;

    /**
     * Sets an item in cache
     * @param key Cache key
     * @param value Value to cache
     */
    set(key: string, value: T): Promise<void>;
    delete(key: string): Promise<void>;
  }

  /**
   * A binary cache provider allows writing and reading binary data to cache (images, blobs, etc.).
   */
  export type BinaryCacheProvider = CacheProvider<Uint8Array>;

  /**
   * A CacheAdapter is the actual adapter in charge of interacting with the underlying storage.
   * This only is used by integrators wanting to provide their own cache adapters.
   */
  export interface CacheAdapter {
    /**
     * Verifies if a namespace value is valid for this cache adapter.
     *
     * @param {string} namespace Cache Adapter
     * @returns {boolean} True if the namespace value conforms to the limitations of this cache adapter.
     */
    isValidNamespace(namespace: string): boolean;
    /**
     * Verifies if a key value is valid for this cache provider.
     *
     * @param {string} key Cache Key
     * @returns {boolean} True if the key value conforms to the limitations of this cache adapter.
     */
    isValidKey(key: string): boolean;

    /**
     *
     * @param namespace
     * @param key
     */
    get(namespace: string, key: string): Promise<Uint8Array>;
    has(namespace: string, key: string): Promise<boolean>;
    set(namespace: string, key: string, value: Uint8Array): Promise<void>;
    delete(namespace: string, key: string): Promise<void>;
  }

  /**
   * Internal helper type to determine if a type can be represented as JSON
   */
  type Serializable<T> = T extends string | number | boolean | null
    ? T
    : // eslint-disable-next-line @typescript-eslint/ban-types
    T extends Function
    ? never
    : T extends object
    ? { [K in keyof T]: Serializable<T[K]> }
    : never;
}

Here is an example of a usage of a cache in user space. Imagine an endpoint that generates OpenGraph share images from content collection entries. Automatically caching the result of the generation during a build becomes trivial.

[slug]-share.png.ts

import type { APIContext, GetStaticPathsResult } from "astro";
import { getCollection, type CollectionEntry } from "astro:content";
import { generateCacheKey, getCache } from "astro:cache";

import { generateShareImage } from "../generators/share-image";

const cache = getCache("posts-share-images");

export async function getStaticPaths(): Promise<GetStaticPathsResult> {
  const posts = await getCollection("posts");

  return posts.map((post) => ({
    params: { slug: post.slug },
    props: { post },
  }));
}

export async function get({ props }: APIContext) {
  const post: CollectionEntry<"posts"> = props.post;

  // Create cache key
  const cacheKey = await generateCacheKey(post, { suffix: ".png" });

  // Get from cache, or call the factory for the image if not present in the cache.
  const imageData = await cache.getOrCreate(cacheKey, () =>
    generateShareImage(post),
  );

  // Return the response
  return new Response(imageData);
}

Astro would provide two adapters by default... A nodeFsCacheAdapter and a memoryCacheAdapter.

In astro.config.mjs, adapters could be configured per-namespace.

import { defineConfig, nodeFsCacheAdapter, memoryCacheAdapter } from 'astro/config';

export default defineConfig({
  caching: {
    defaultAdapter: nodeFsCacheAdapter,
    namespaces: {
      myCustomNamespace: memoryCacheAdapter,
    },
  },
});

getCache and getValueCache would return the proper cache provider facade with the right adapter based on the configuration.

[spaceemotion]

Not sure why this hasn't been discussed further - we came across the need for better caching this month as well. We're generating images and data asynchronously when a page loads. This isn't quite the issue when statically building the site, but it does make the build process longer, as well as local development.

Having a way to say "based on the input, tell me if we need to recalculate stuff - otherwise, use the output from last time" would be very helpful and save tons of time and resources (our website now takes multiple minutes to build, including reusing the image cache).

[markomitranic]

This is definitely a valuable thing to have, and a great proposal in terms of its interface.

I would suggest looking into integrating a well-known, standardized cache library instead. For example NestJS uses Cacheable which is a very mature implementation of (already mature) keyv cache adapters for different stores. It not only gives us all the adapters for different stores out of the box, but also has built-in stampede prevention, L1&L2 cache, cache nesting, revalidation, and many other niceities. NestJS relies on it A LOT, and managed to "wrap" it in different ways, to offer composite features like queues, remote procedure calls, blob cache etc. In fact it also is becoming a popular replacement for unstable_cache, in the Next.JS ecosystem, as their cache is crap, I use it with an in-memory LRU L1 cache and Upstash Redis as L2 cache, and use it both for data and component/page caching.

My suggestion isn't to limit our interfaces in any way, but instead to take what is already there as a solid base, for writing our own wrappers, much like Nest did.

[stelcodes]

It would be nice to have this cache available to remark/rehype components as well. I'm using a Remark plugin to generate GitHub cards via the GitHub API, and I'd like to cache the JSON responses to avoid being rate limited. This is especially problematic while using the development hot-reloading server because editing markdown with a GitHub card sends many requests in a short amount of time.

[LasseRosenow]

I also need this feature a lot.
My use case is that my websites integrate with a CMS and I am rebuilding the page when content in the CMS changes.

Right now I have a build time of 100 seconds which is mostly just the fetching of images for all of the many pages even though they are already cached.
I would like to prevent the fetches being done based on a cache key, which in my case would contain the URL and the last modified timestamp.
This way it is not necessary to fetch the image to find out, that it has not changed.

[molszanski]

Oh you have SSG I guess. Never mind previous comment

[LasseRosenow]

Oh you have SSG I guess. Never mind previous comment

Actually I don't have SSG,Yes I do have SSG, I automatically rebuild and redeploy my page if content in the CMS changes. I think the static site generator is great because it lets me save a lot of memory consumption compared to SSG SSR, which needs a node.js runtime. :)

[LasseRosenow]

Maybe use any third party solution to store the data locally? People often use Redis or memcached for that but IMO any ram / disk based cache from nom would be good enough

This probably works, but I still would prefer to be able to just give a key and a value to the astro build-in cache and let them integrate it nicely into their build pipeline instead of rolling my own workaround :)

[stelcodes]

@LasseRosenow I think y'all are actually talking about the same thing? SSG == static site generation. Maybe you're thinking of SSR? Anyway, it would be awesome to have a simple filesystem cache for this use case.

[LasseRosenow]

@LasseRosenow I think y'all are actually talking about the same thing? SSG == static site generation. Maybe you're thinking of SSR? Anyway, it would be awesome to have a simple filesystem cache for this use case.

Oh sorry yes I somehow got confused sorry sorry :)
I indeed mean the static site generation :)

[FineWolf]

To clear some confusion here... I'm the author of this proposal.

The reason why I suggested this is because, in my current SSG scenario for my website, I needed a cache to speed up building. I ended up implementing my own.

I have various use-cases on my own website where I use it:

• Generating thumbhashes for image assets
• Determining the dominant colour for image assets
• Dynamically generated social share images
• Extracting EXIF metadata from image assets

Without caching those, builds can take upwards of 5 minutes. With caching, subsequent builds are done in less than 20 seconds.

The point of this proposal is to have a standard interface for this kind of work, with a default implementation that uses the filesystem. If you want to replace the implementation to use a key store, you'll be free to do so.

However, in my SSG scenario, I do not want to rely on external infrastructure. My whole reason why I'm using Astro is that I can build my website and deploy on cheap blob-based hosting with minimal infrastructure, and not have to worry about scalability.