Comments in .tmpl are evaluated

[petabook]

Describe the Bug

Atmos version v1.195.0.
This new version caught an error in one of my .tmpl files that previous versions missed - so far, so good.
However, after commenting out the problematic line, it still reports the same error. It only stops complaining once the line is completely removed.

catalog/qh/di/aws-tgw-attachment/secondary.yaml.tmpl

components:
  terraform:
    "tgw-attachment/{{ .name }}":
      metadata:
        component: cloudposse/aws-tgw-attachment
      vars:
        name: "{{ .name }}"
        vpc_component_name: "vpc/{{ .name }}"
        transit_gateway_id: !terraform.state tgw-hub transit_gateway_id
        transit_gateway_route_table_id: !terraform.state tgw-hub transit_gateway_route_table_id
        # transit_gateway_attachment_subnet_name: '{{ default "" .transit_gateway_attachment_subnet_name }}'

The error

Executing command: `atmos terraform deploy cloudposse/aws-team-roles -s core-gbl-root`

 Error

 invalid stack manifest: template: catalog/qh/di/aws-tgw-attachment/secondary.yaml.tmpl:11:65: executing "catalog/qh/di/aws-tgw-attachment/secondary.yaml.tmpl" at <.transit_gateway_attachment_subnet_name>: map has no entry for key "transit_gateway_attachment_subnet_name"

 File being processed: catalog/qh/di/aws-tgw-attachment/secondary.yaml.tmpl
 Import chain:
   → orgs/qh/prod/sanofi/eu-west-3/network.yaml
     → catalog/qh/di/network-secondary.yaml
     → catalog/qh/di/aws-tgw-attachment/secondary.yaml.tmpl
 stack manifest 'catalog/qh/di/aws-tgw-attachment/secondary.yaml.tmpl'

Expected Behavior

Ignore comments.

Steps to Reproduce

Provided above.

Screenshots

No response

Environment

No response

Additional Context

No response

[osterman]

Thanks for the report!

Are you sure this behavior wasn’t already present before? Fundamentally, .tmpl files are processed as plain text first — they’re treated as Go templates, not YAML. There’s no concept of YAML at that stage (and can’t be), so every line, including comments, is evaluated as part of the template.

Only after that step is the rendered output parsed as YAML. That’s how it (should have) always worked, so I’m skeptical that anything has changed here.

Also note that importing a .tmpl file is different from the YAML templating mechanism we use in stack configurations elsewhere in Atmos.

[aknysh]

this commented line is also executed as a Go template:

# transit_gateway_attachment_subnet_name: '{{ default "" .transit_gateway_attachment_subnet_name }}'

b/c as Erik mentioned, the templating engine does not know anything about YAML comments in template files, it will find templates everywhere in the file.

If that was working before, it's strange.
@petabook can you please confirm it on the prev version if you get a moment?

[rowkv]

atmos@1.195.0
catalog/dns-delegated/defaults.yaml.tmpl

components:
  terraform: 
    dns-delegated:
      vars:
        zone_config:
          - subdomain: "{{ .vars.stage }}.{{ .vars.tenant }}"
            zone_name: example.com
        request_acm_certificate: false
        dns_private_zone_enabled: false

Error:

 invalid stack manifest: template: catalog/dns-delegated/defaults.yaml.tmpl:6:32: executing "catalog/dns-delegated/defaults.yaml.tmpl" at <.vars.stage>: map has no entry for key "vars"

 File being processed: catalog/dns-delegated/defaults.yaml.tmpl
 Import chain:
   → orgs/example/sandbox/eng/global-region.yaml
     → catalog/dns-delegated/defaults.yaml.tmpl
 stack manifest 'catalog/dns-delegated/defaults.yaml.tmpl'

atmos@1.194.0 breaks on vars with !terraform.state usage

atmos@1.193.0 works fine

[j4zzcat]

I can confirm that this is working in 1.189.0 and fails, perhaps as expected per your analysis, in 1.195.0 and 1.198.0.

[aknysh]

atmos@1.195.0 catalog/dns-delegated/defaults.yaml.tmpl

components:
terraform:
dns-delegated:
vars:
zone_config:
- subdomain: "{{ .vars.stage }}.{{ .vars.tenant }}"
zone_name: example.com
request_acm_certificate: false
dns_private_zone_enabled: false
Error:

 invalid stack manifest: template: catalog/dns-delegated/defaults.yaml.tmpl:6:32: executing "catalog/dns-delegated/defaults.yaml.tmpl" at <.vars.stage>: map has no entry for key "vars"

 File being processed: catalog/dns-delegated/defaults.yaml.tmpl
 Import chain:
   → orgs/example/sandbox/eng/global-region.yaml
     → catalog/dns-delegated/defaults.yaml.tmpl
 stack manifest 'catalog/dns-delegated/defaults.yaml.tmpl'

atmos@1.194.0 breaks on vars with !terraform.state usage

atmos@1.193.0 works fine

@rowkv thanks for reporting. Please provide more info on the issue:

• atmos@1.194.0 breaks on vars with !terraform.state usage. - how it's related to !terraform.state ? (it's not in your example)
• If you import catalog/dns-delegated/defaults.yaml.tmpl, then the error is correct. Importing a stack manifest will process Go templates. See these docs:
  • https://atmos.tools/core-concepts/stacks/imports/#go-templates-in-imports
  • https://atmos.tools/core-concepts/stacks/imports/#imports-with-templates
  • https://atmos.tools/core-concepts/stacks/templates/#excluding-templates-in-imports
  • https://atmos.tools/core-concepts/stacks/templates/#phases-of-template-evaluation
    Importing a Go template will evaluate the templates, and for it to work, it requires context. Or, if you want to import it and don't process the templates during importing, but process them at a later stage (when the component in the stack is found and the vars section already known, you need to do what is described in https://atmos.tools/core-concepts/stacks/templates/#excluding-templates-in-imports

Let me know what is your config

[rowkv]

@aknysh ok, last doc gave me better understanding of what's going on.

I've been testing on atmos@1.198.0 since didn't have much time to check all versions.
Following config works.

components:
  terraform: 
    dns-delegated:
      vars:
        zone_config:
          - subdomain: "{{`{{ .vars.stage }}`}}.{{`{{ .vars.tenant }}`}}"
            zone_name: example.com
        request_acm_certificate: false
        dns_private_zone_enabled: false

I also had another template which uses default function, so it needed adjustments as well.

atmos.yaml

templates: 
  settings:
    enabled: true
    sprig:
      enabled: true

FROM

components:
  terraform:
    {{ range $i, $instance := .instances }}
    {{ $instance.component_name }}:
      metadata:
        component: ec2-instance
      vars:
        ...
        enabled: "{{ $.enabled | default true }}"
        ...
    {{ end }}

TO

components:
  terraform:
    {{ range $i, $instance := .instances }}
    {{ $instance.component_name }}:
      metadata:
        component: ec2-instance
      vars:
        ...
        enabled: '{{ printf "{{ $.enabled | default true }}" }}'
        ...
    {{ end }}

I still had some weird issues with vars on 1.194.0, while 1.198.0 works fine.

[mss]

I think the proper title for this issue would be "default function doesn't work anymore with partial context".

We also ran into this issue with a file which looks like this (trimmed down):

import:
  - path: catalog/monitoring/v0_2
    context:
      monitoring_type: wms

And the first two lines of that tmpl file looking like this (the error is triggered in line 2):

import:
  - path: ./mixins/{{ .monitoring_stage | default "production" }}

Ie. we do provide a context but just a partial one and use default to fill in all the missing fields. This worked fine until v1.195.0 and is still broken in v1.199.0. It looks like until that version the map access behaviour was changed from returning a nil or empty string to raising an error instead.

I think we should be able to change that line to something like

import:
  - path: ./mixins/{{ with index . "monitoring_stage" }}{{ . }}{{ else }}production{{ end }}

The default approach was nicer to read though so I wonder if this is considered a bug and will be fixed?