Skip to content

Add a utility to get a local directory that can be used as a persistent cache #1053

@llucax

Description

@llucax

What's needed?

We need to be able to access some directory that can be used to store cache data that can persist application restarts or re-deployment.

Proposed solution

Add an utility function like def get_cache_dir(unique_cache_key: str) -> pathlib.Path. Where this utility function should live is still to be determine.

For now the SDK should have some initialization parameter called cache_dir_root or similar, and/or being able to get this configuration from some SDK_CACHE_DIR_ROOT (we could also have some fallback default using the XDG standard, like $XDG_CACHE_HOME).

Use cases

A typical use case is to cache data that actors or applications need for startup. For example, to do some predictions we need to collect data for some time and after enough data is collected, we can start predicting. If the application is restarted, this data collection period kicks in again, which can leave the app effectively not doing its job for a non trivial amount of time (warm-up period).

By using a cache, we can save the current status and pick up right were we left it when restarting, so we can skip this warm-up period.

Alternatives and workarounds

No response

Additional context

For cloud apps we might eventually want to use something else than a raw filesystem, like a object store (like AWS S3 compatible storage) for scalability reasons. If we move in that direction, this approach will not be enough, as apps/actors will need to act on a per-file basis, explicitly saving and retrieving from the object store.

Metadata

Metadata

Assignees

No one assigned

    Labels

    part:coreAffects the SDK core components (data structures, etc.)priority:❓We need to figure out how soon this should be addressedtype:enhancementNew feature or enhancement visitble to users

    Type

    No type

    Projects

    Status

    To do

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions