Restricting what modules can be loaded into the runtime

[knolleary]

I'm working on the Plugins design work and have identified a need to restrict what modules the runtime is allowed to load. This is similar to the nodesExcludes property, but works at a module name level.

As I wrote out a design for it, I realised I was reproducing what is already proposed in the Function Lifecycle design from @HiroyasuNishiyama .

So I wanted to start a discussion (Hey - using the newly available Discussions feature on GitHub!) around how we can make this setting more generally used. I think we have a few different use cases for installing modules in Node-RED and want to make sure the settings we provide are logically consistent across them all.

Current settings

We currently provide the following settings related to installing new things in Node-RED:

• editorTheme.palette.editable - if set to false, the Admin API for installing/removing nodes is disabled and the editor hides the palette manager
• nodesExcludes - an array of individual filenames the runtime will refuse to load
• autoInstallModules - if set to true, the runtime will attempt to automatically reinstall node modules it had listed in its runtime settings (.config.nodes.json), but didn't find on startup.

Existing designs

We have three different designs in progress that relate to installing new modules.

1. Support for Subflows published as npm modules
2. Support for Function nodes to specify any npm module dependency

   externalModules: {
       mode: "manual",
       allowList: [
           "^fs-ext$",       // allow fs-ext
           "^[email protected]$"  // allow qrcode version 1.4.4
       ],
       denyList: [
           ".*"              // disallow others 
       ]
    },
   

3. Support for Plugins that are published as npm modules

We must make sure we have a consistent group of settings across all of these features.

Requirements

The main requirement is to provide the Node-RED Administrator (the person who edits the settings file) the ability to keep control over what a Node-RED User (the person using the editor) is able to install.

That should include:

• whether the runtime will try to automatically install anything
• whether the user should be allowed to try to install anything (ie, whether the Editor should even show any UI features related to installing nodes/modules)
• providing an allow/deny list of modules to restrict what a user can try to install

Proposed settings

I propose we have a single setting block that covers installing modules in general (nodes, Function npm dependencies, plugins). Basically anything that involves running npm install.

externalModules: {
   allowInstall: true/false,   // Whether the ability to install/remove modules is enabled
   autoInstall: true/false, // Whether missing modules will be automatically installed
   allowUpload: true/false,
   allowList: [],
   denyList: []
}

We will then deprecate the existing settings as follows:

• Deprecate editorTheme.palette.editable for externalModules.allowInstall
• Deprecate autoInstallModules for externalModules.allowInstall = true and externalModules.autoInstall = true
• Deprecate editorTheme.palette.upload for externalModules.allowUpload

The allow/deny lists will use simple globs rather than full regexes to keep it simple for the user.

For example:

 allowList: [
  "fs-ext",       // allow fs-ext at any version
  "[email protected]",  // allow qrcode version 1.4.4
  "@acme/*" // allow anything published by the `@acme` organisations
]

If allowList is set, then denyList will default to ["*"] unless otherwise set.
If denyList is set, then allowList will default to ["*"] unless otherwise set.

The version specifier in the glob can use <, >, <=, >= and ~ in accordance with normal npm matching rules.

The allowList/denyList values will be passed to the editor so it can filter the list of modules it shows in the palette manager install tab.

auto-update not included

This proposal does not include the auto-update feature proposed in the Function node lifecycle design. I do not think the runtime should ever silently upgrade modules. That should be something a user/administrator chooses to do.

[knolleary]

I'm going to be implementing this as outlined above as part of the plugins work. Would be great to get some feedback before I spend too much time on it.

[HiroyasuNishiyama]

Seems good design proposal also for NPM module installation for function node.
Some questions and comments:

• Are the default value as follows?

externalModules: {
   allowInstall: true,
   autoInstall: false,
   allowUpload: true,
   allowList: ['*'],
   denyList: []
}

• It would be nice if API or utility functions will be defined for accessing this information.
• It would be nice to be able to specify an array of specification objects with a specifier (e.g. kind: "plugin") as the value of externalModule so that the settings can be separated for each purpose.

[knolleary]

Are the default value as follows?

Yes.

It would be nice if API or utility functions will be defined for accessing this information.

I am assuming there is a central component of the runtime that handles installing modules of any type - so all of this information will be used by that one component, which I believe reduces the need for an internal API to expose this information. That said, it will be accessible to the whole runtime under RED.settings as all runtime settings are.

The question for the Function node module work is what changes are needed to the existing @node-red/registry/installer.js code to be able to support the non-node/plugin use cases - and how that is exposed in the runtime. But that's a discussion to have over in #31 - will add some comments over there.

I am going to start implementing the handling for this logic in the existing registry/installer code as part of the plugins branch.

[knolleary]

It would be nice to be able to specify an array of specification objects with a specifier (e.g. kind: "plugin") as the value of externalModule so that the settings can be separated for each purpose.

I've been thinking about this. I'm not sure I can see the need for it. All modules are installed via npm - having a module on a denyList as a plugin but allowed as a node doesn't not seem a useful distinction. Is there a particular scenario you have in mind here?

[HiroyasuNishiyama]

I thought that some users may allow nodes to be installed, but may not allow plugins and NPM modules for Function nodes.
In such case, because allowInstall is single global setting, it seems that we need to devise the value of allowList and/or denyList.

[knolleary]

Ah yes, of course... I forgot about that part of it.

Yes, we need a way to enable/disable palette manager and enable/disable module manager as separate things.

Maybe going back to having separate palette and external module configs is better... Need to think about it some more. Will update tomorrow.

[knolleary]

Found another setting to deprecate - autoInstallModulesRetry - a time in ms before retrying a failed auto install. It defaults to 30000.

This will be replaced by autoInstallRetry - and I propose to change the units to seconds. There is no good reason to have it as milliseconds.

externalModules: {
   allowInstall: true,   // Whether the ability to install/remove modules is enabled
   autoInstall: false, // Whether missing modules will be automatically installed
   autoInstallRetry: 30,
   allowUpload: true,
   allowList: ['*'],
   denyList: []
}

[knolleary]

As @HiroyasuNishiyama points out, we need to separate the enabling of installing new nodes from installing new modules.

So instead of the single allowInstall, I propose:

externalModules: {
   allowPaletteInstall: true,
   allowModuleInstall: true, 
   autoInstall:false,
   allowPaletteUpload: true,
   allowList: [],
   denyList: []
}

• allowPaletteInstall - determines whether the palette manager is enabled and the corresponding Admin APIs
• allowModuleInstall - determines whether the module manager UI is enabled and the corresponding Admin APIs
• allowPaletteUpload - previously allowUpload, renamed to be clear this is referring to node modules only

The other properties remain the same and apply consistently across both nodes and modules. I don't see why having separate allow/denyList properties is needed.

[knolleary]

Okay - I got this wrong. @HiroyasuNishiyama you were right to suggest having the settings scoped to the different purposes.

I wanted to have a single set of properties that applied to all cases. But as I worked through it, I can see problems with that approach. In particular, it is the allow/denyList properties that cause problems if we only have the one list.

A reasonable scenario is a Node-RED administrator was to allow any node to be added to the palette, but to restrict the modules available to the Function node to a smaller list. With a single allows/deny list, it would be nearly impossible to achieve that... you'd need the allowList to contain all possible node modules.

So what do we think about the following:

externalModules: {
    autoInstall: true/false,
    autoInstallRetry: 30,
    palette: {
        allowInstall: true/false,
        allowUpload: true/false,
        allowList: [],
        denyList: []
    },
    modules: {
        allowInstall: true/false,
        allowList: [],
        denyList: []
    }
}

[HiroyasuNishiyama]

I thought that restricting only NPM modules for function node may be achieved using glob pattern such as node-red-* after looking your proposal.
But, yes, separating specification for each purpose seems better approach.

[knolleary]

Have raised a PR against Node-RED core that implements the externalModules setting. This does not cover the externalModules.modules.* part of the design as that will come with the Function node work.

node-red/node-red#2797