VRL doesn't honor `--color=never`

[mmelvin0]

Vector Version

vector 0.13.1 (v0.13.1 x86_64-pc-windows-msvc 2021-04-29)

Vector Configuration File

I don't think anything in the configuration is relevant, but just in case:

https://gist.github.com/mmelvin0/d2d532ad3b13907753f6a6db781a4047

The above is a working configuration file. To reproduce the error, introduce any Vector Remap Language compile-time error.

Debug Output

The color escapes can be seen in the debug output:

https://gist.github.com/mmelvin0/481ad49544e141203e4ca1aa86c200b8

Expected Behavior

To be able to easily read Vector's JSON log output when running as:

vector -vvv --color=never --log-format=json --config=vector.toml

Actual Behavior

JSON logs about configuration errors are difficult to read because they contain embedded color escape sequences.

Example Data

{"timestamp":"May 23 12:33:31.021","level":"ERROR","message":"Configuration error.","error":"Transform \"remap_fluentbit\": \n\u001b[0m\u001b[1m\u001b[38;5;9merror[E651]\u001b[0m\u001b[1m: unnecessary error coalescing operation\u001b[0m\n  \u001b[0m\u001b[36m┌─\u001b[0m :8:4\n  \u001b[0m\u001b[36m│\u001b[0m\n\u001b[0m\u001b[36m8\u001b[0m \u001b[0m\u001b[36m│\u001b[0m if \u001b[0m\u001b[31mparts.lvl != null\u001b[0m ?? false {\n  \u001b[0m\u001b[36m│\u001b[0m    \u001b[0m\u001b[31m^^^^^^^^^^^^^^^^^\u001b[0m \u001b[0m\u001b[36m--\u001b[0m \u001b[0m\u001b[36m-----\u001b[0m \u001b[0m\u001b[36mthis expression never resolves\u001b[0m\n  \u001b[0m\u001b[36m│\u001b[0m    \u001b[0m\u001b[31m│\u001b[0m                 \u001b[0m\u001b[36m│\u001b[0m\n  \u001b[0m\u001b[36m│\u001b[0m    \u001b[0m\u001b[31m│\u001b[0m                 \u001b[0m\u001b[36mremove this error coalescing operation\u001b[0m\n  \u001b[0m\u001b[36m│\u001b[0m    \u001b[0m\u001b[31mthis expression can't fail\u001b[0m\n  \u001b[0m\u001b[36m│\u001b[0m\n  \u001b[0m\u001b[36m=\u001b[0m see language documentation at https://vrl.dev\n","target":"vector::topology"}

Additional Context

I'm running Vector on Windows 10 as a service under NSSM. My use case is to be able to read the JSON log in case of a configuration error in vector.yaml when I restart the service. I'd like to avoid popping open PowerShell to debug Vector configuration.

References

The --color CLI argument was originally added in #599.

Arguably JSON output should never contain color escape sequences, even if the output is a terminal.

[jszwedko]

Thanks @mmelvin0 . I can see this is the VRL error output that is still colored in this case.

[logabot]

I catch "same" error, bit this validate. If validate is success, --color never works fine,

~# vector -V
vector 0.19.1 (x86_64-unknown-linux-gnu 3cf70cf 2022-01-25)
~# vector --color never validate --config-yaml /etc/vector/vector.yaml
√ Loaded ["/etc/vector/vector.yaml"]
2022-01-31T12:47:50.412836Z  WARN vector::transforms::regex_parser: Usage of `regex` is deprecated and will be removed in a future version. Please upgrade your config to use `patterns` instead: `patterns = ['^\[(?P<timestamp>[\w\-:\+]+)\] (?P<level>[\w]+): (?P<message>.*)$']`. For more info, take a look at the documentation at https://vector.dev/docs/reference/transforms/regex_parser/.
√ Component configuration
√ Health check "vector_exporter"
√ Health check "vector_tmp"
~ Health check disabled for "to_external_vector"
------------------------------------------------
                                       Validated

But when validate is error, --color never ignoring

 ["2022-01-31T12:44:15.277532Z  WARN vector::transforms::regex_parser:
 Usage of `regex` is deprecated and will be removed in a future version. 
Please upgrade your config to use `patterns` instead: 
`patterns = ['^\\[(?P<timestamp>[\\w\\-:\\+]+)\\] (?P<level>[\\w]+): (?P<message>.*)$']`. 
For more info, take a look at the documentation at https://vector.dev/docs/reference/transforms/regex_parser/."],
"stdout": "√ Loaded [\"/root/.ansible/tmp/ansible-tmp-1643633053.051413-26831-129274333956942/source\"]\n\n
Component errors\n----------------\nx Transform \"parse_nginx_error_log\": 
\n\u001b[0m\u001b[1m\u001b[38;5;9merror[E701]\u001b[0m\u001b[1m: call to undefined variable\u001b[0m\n  
\u001b[0m\u001b[34m┌─\u001b[0m :1:35\n  \u001b[0m\u001b[34m│\u001b[0m\n\u001b[0m\u001b[34m1\u001b[0m
 \u001b[0m\u001b[34m│\u001b[0m . = merge(object(parse_nginx_log!(\u001b[0m\u001b[31mmessage\u001b[0m, 
\"error\")), . )\n  \u001b[0m\u001b[34m│\u001b[0m                                   \u001b[0m\u001b[31m^^^^^^^\u001b[0m\n
  \u001b[0m\u001b[34m│\u001b[0m
...etc

Create new issue or stay here?