Pipeline Config file string validation is too strict for values in load_from_yaml()

[mwade-noetic]

I am building custom pipelines & nodes and putting various information for the nodes in the PARAMS of the pipeline.yaml file.
Recent updates have added key & value validation, but the latest (1.9.1) validation for values prevents common scenarios:

For example, having a parameter:
username: [email protected]

This is not accepted as a valid value and throws an exception. The regex used: '^[-\w/\.:* []]+$' for this validation is pretty strict. I am not sure what problem you are specifically trying to avoid with this check, but adding support for standard punctuation characters and things like @ or # would be most helpful.

Is there any work-around other than setting environment variables? (Still need to confirm that this actually works when the over-ride is enabled.

[ZanSara]

Hey @mwade-noetic! Right now there's no workaround for this, but let's convert this discussion into an issue to track the problem. You're not the only one that faced a problem like this.

[mwade-noetic]

I think the key question is what are you trying to prevent with the validation that is taking place on the value side. The regex to avoid certain characters (punctuation, and other items) seems to indicate the goal was to reduce down to only a few very specific types of strings being passed in. However, given the hack with the CUSTOM_QUERY (JSON type data) I don't believe this is really useful. My vote would be to leave it to the node / pipeline to validate the values.

I see you are already loading from the yaml file using safe safe loads so that provides some protection for security issues.