Gomplate produce output file when parsing fails

[valepert]

Subject of the issue

gomplate using an input template with reference to an environment variable that doesn't exist, still writes an output file.

Your environment

• gomplate version 3.5.0

Steps to reproduce

$ gomplate -i 'fail: {{ .Env.BOGUS }}' -o fail.yml
template: <arg>:1:13: executing "<arg>" at <.Env.BOGUS>: map has no entry for key "BOGUS"

Expected behaviour

$ stat -f "%N" fail.yml
stat: fail.yml: stat: No such file or directory
$ test -f fail.yml || echo $?
1

Actual behaviour

$ stat -f "%N" fail.yml
fail.yml
$ test -f fail.yml && echo $?
0

[hairyhenderson]

Hi @valepert, thanks for logging this, and sorry for the slow response - I've been on vacation 😉

This is the expected behaviour. Because the template is parsed and interpreted simultaneously, gomplate creates the output file before it starts to render the template.

Also, this is a useful behaviour when troubleshooting template errors, especially with larger templates, since the content that successfully rendered will all be present in the output.

One way to accomplish what you're looking for is to delete the file on failure:

$ gomplate -i 'fail: {{ .Env.BOGUS }}' -o fail.yml || rm -f fail.yml

This way, fail.yml will never be present when gomplate fails for any reason.

A slightly different approach, without requiring a shell, would be to try to resolve environment variables up front, and then use the GOMPLATE_SUPPRESS_EMPTY environment variable to cause the output file to not be written on failures:

$ GOMPLATE_SUPPRESS_EMPTY=true gomplate -i '{{ $bogus := .Env.BOGUS -}}
fail: {{ $bogus }}' -o fail.yml
template: <arg>:1:17: executing "<arg>" at <.Env.BOGUS>: map has no entry for key "BOGUS"
$ ls fail.yml
ls: cannot access 'fail.yml': No such file or directory

This works because the template fails before any output has been rendered.

I'm going to close this issue now, since this is working as designed. Feel free to re-open though, if you feel strongly about this, or if the suggested workarounds don't work 😉

[ProvoK]

Hi @hairyhenderson
I think your approach is solid and maybe this conversation could be used to improve the actual documentation.

I still think the issue persist because the output (file or stdout) is a stream. This is good from a Unix philosophy, don't get me wrong, but this make quite difficult to use gomplate with pipes.

For instance:

# file: fail.tpl
Hello {{ .Env.USER }}
Today is {{ .Env.BOGUS }}

the partially rendered output will be passed to the pipe, regardless it fails or not

$ gomplate -f fail.tpl 2>/dev/null| xargs echo

Hello vic Today is

In one side project of mine, I'm using gomplate to render a config file for my application because it may contains secrets (access keys, apikeys etc).
In this case, I'd like to don't write sensible information into a file (so the "--" approach does not work for me) and my application consumes the config from stdin (it has also a flag for the config path, of course, is security is not a concern).

The workaround I'm using is the following

$ rendered_template=$(gomplate -f template.tpl) && echo "$rendered_template" | my_binary

It works, and it's fine.
It's kinda verbose though, and difficult to read.
I'm not sure about the flag name, but something like the following may be more clean and easy to reason about

$ gomplate -f template.tpl -stdin -- my_binary

where stdin stands for "pass as input to the command I'll execute on success"

It's not a blocking issue, and there are plenty of workarounds.
But I still think that is something quite important and will come often out in the future.

I could try to do a PR to extend the documentation, if you like

EDIT: After some thinking, I realized that the fact that the gomplate "stream" could be incomplete may be not very adherent to Unix philosophy

[hairyhenderson]

This is good from a Unix philosophy, don't get me wrong, but this make quite difficult to use gomplate with pipes.
[...]
EDIT: After some thinking, I realized that the fact that the gomplate "stream" could be incomplete may be not very adherent to Unix philosophy

Though I'm generally a fan of the Unix philosophy, I've never made any special effort to make gomplate with it... So any similarities are purely coincidental and unintentional 😉

What may be possible is a feature like the GOMPLATE_SUPPRESS_EMPTY one, where you could set an environment variable to change its behaviour to buffer rendering and not write to the output stream when errors are encountered. The challenge is what to do when outputting multiple files - I suppose it would have to output file-by-file...

Or maybe the behaviour could be changed in the next major version of gomplate (v4) 🤔...

I'm going to re-open this since I'm now thinking this may be a good change in default behaviour, though until the next major release it would be opt-in. The tradeoff (slower time to any data being written) is probably worth it!

[hairyhenderson]

I could try to do a PR to extend the documentation, if you like

@ProvoK that would be great! perhaps it would be good to also mention that the behaviour will change in future versions.

[hairyhenderson]

I'm not sure about the flag name, but something like the following may be more clean and easy to reason about

$ gomplate -f template.tpl -stdin -- my_binary

where stdin stands for "pass as input to the command I'll execute on success"

I'd thought about that when I wrote that feature, and opted to ignore it at that point - but I'm definitely open to it! Can you open a separate issue for that? 🙂

[ProvoK]

I could try to do a PR to extend the documentation, if you like

@ProvoK that would be great! perhaps it would be good to also mention that the behaviour will change in future versions.

I'm gonna do it 😄 Probably next week