bloblang: format string manipulation method percent escape

[gramian]

Dear Developers, Dear Community,

the documentation of the format method does not state how to escape a literal percent % (see: https://www.benthos.dev/docs/guides/bloblang/methods/#format ). I assume a double percent %% can be used for this, right? Could you extend the documentation with this information?

Furthermore, if a URL with a query string needs formatting, many percent symbols may need to be escaped. Thus, I would like to ask if the formatting symbol could be made customizable (with a default being the percent symbol)?

Thank You

[mihaitodor]

Hey @gramian, thanks for pointing that out! I added some extra information to the format method docs in #1728 to link the Go documentation of the fmt package which lists the valid verbs (as well as the fact that %% resolves to a literal percent sign.

While I can see that in some situations it might be desirable to use another character instead of % to specify verbs, it will introduce extra complexity which can be confusing and error-prone. I think calling something like replace_all("%", "%%") should be acceptable in most cases. Please share a detailed example if you're aware of a situation where this doesn't work well.

[gramian]

Hi @mihaitodor ,
thanks for your answer. With replace_all this can be done in two steps. Sorry, I did not think of that.

Since you asked, here is a bloblang string operation which is currently done with concatenations for which I would rather do format:

SELECT%20name%2C%20publicationYear%2C%20%40rid%20FROM%20dataset%20WHERE%20resourceType%20IN%20%5B" + meta("resourceType").join("%2C") + "%5D%20AND%20subject%20IN%20%5B" + meta("subject").join("%2C") + "%5D%20AND%20language%20IN%20%5B" + meta("language").join("%2C") + "%5D%20AND%20license%20IN%20%5B" + meta("license").join("%2C") + "%5D%20ORDER%20BY%20name%3B"

Here, for a static SQL-type query encoded as URL (with many %s), certain parts are replaced with input URL parameters.

[mihaitodor]

Thanks! If you have a lot of those, it's probably best to add a custom plugin (see the bloblang folder) which exposes a new method for advanced formatting. There might be some 3rd party formatting library which lets you customise the verb specifier, since the builtin one doesn't seem to allow this and doing a string replace on the format string is error-prone.