Storage API v2

[knolleary]

The Storage API is one of the oldest APIs in Node-RED. It works well for general single-user/single-instance versions of Node-RED, but it also limits what we're able to do in the core given a number of features the API is missing.

This discussion is a place to gather requirements for a v2 of the API.

To help structure the requirement gathering, I'm going to add a reply for each major functional area of the existing API. Please add comments to the individual replies.

[knolleary]

Flow Storage

[knolleary]

Version Control

The Projects features exists in the Local File System storage plugin and works as an undocumented extension to the API. In theory, another storage plugin could implement the huge surface area of the projects API. But the API is very much a git-influenced api.

I think there is a need to look at this again - and to see if there is a simpler api to provide basic version control without necessarily assuming a git repo. It should not stop a plugin providing a more advanced api to provide the full capabilities of projects - but being able to do basic versioning of flows should be possible.

[knolleary]

Storage->Runtime comms

The current API only allows the runtime to call the storage module. There is no way for the storage module to communicate back to the runtime. For example, in an HA scenario, the storage module may need to notify the runtime that its flows have been updated. The new API needs to provide a mechanism for storage-initiated events.

[knolleary]

Core Library Storage

[knolleary]

We added Library plugins in the last release. Does it make sense to have the core library still part of the Storage module api? Or at the very least, there needs to be some code reuse here.

[knolleary]

Settings Storage

[knolleary]

We pile a lot of things into the single settings api. We recently split that out into four .config.*.json files. We should consider whether that split should also be reflected in the api.

[knolleary]

Login Sessions Storage

[knolleary]

Currently the runtime only reads the sessions store when starting up, and writes it as needed. That makes it impossible to pool the session tokens in an HA configuration.

One option would be to have the auth code get the sessions from storage every time it needs to - and rely on the storage plugin doing any cacheing needed to make that work efficiently. For example, a file store could watch the sessions file for changes and re-read it as needed.