Using templatefile directly in a part's content can cause unnecessary plan changes

[euanhunteratom]

Terraform CLI and Provider Versions

Terraform v1.9.1
on linux_amd64

• provider registry.terraform.io/hashicorp/cloudinit v2.3.4
• provider registry.terraform.io/hashicorp/null v3.2.2

Terraform Configuration

# Very contrived minimal reproduction

# main.tf
###
terraform {
  required_version = "1.9.1"

  required_providers {
    cloudinit = {
      source  = "hashicorp/cloudinit"
      version = "2.3.4"
    }
    null = {
      source = "hashicorp/null"
      version = "3.2.2"
    }
  }
}

variable "test_map" {
  type = map(string)
}

data "cloudinit_config" "config" {
  for_each = var.test_map

  part {
    content = templatefile("${path.module}/template.tftpl",{
      value = null_resource.n[each.key].id
    })
  }
}

resource "null_resource" "n" {
  for_each = var.test_map
}

output "config_id" {
  value = data.cloudinit_config.config["itemA"].id
}
###

# template.tfpl
###
${value}
###

# terraform.tfvars
###
test_map = {
  "itemA" = "itemA"
#  "itemB" = "itemA"
}
###

Expected Behavior

Adding a new element to test_map should not cause data.cloudinit_config.config["itemA"] to get read on plan and output config_id should not show as "known after apply", as the template output has not changed.

Actual Behavior

Adding a new element to test_map causes data.cloudinit_config.config["itemA"] to get read on plan and output config_id shows as "known after apply".

The code above is a contrived example but in real world usage a data.cloudinit_config is likely referenced in the user_data field of a cloud instance and so such a change, when the output has not actually changed, leads to unnecessary resource replacement and possibly destructive behaviour.

Steps to Reproduce

1. terraform apply. Note config_id output
2. Uncomment "itemB" in terraform.tfvars
3. terraform plan. Note config_id is now "(known after apply)" despite not changing
4. terraform apply. Note that config_id output is the same as before, indicating nothing actually changed

How much impact is this issue causing?

Medium

Logs

https://gist.github.com/euanhunteratom/e81bc7cba42696779a1fea85a77faa42

Additional Information

Gist log is from the step 3 in the reproduction.

This issue seems to have something to do with the interaction between content, templatefile, and a resource with multiple instances using for_each. The workaround (see below) involves moving the templatefile to a local; and using a resource with only a single instance or static values in the template does not trigger this issue. So, there is something about templatefile being directly used as content where the template references a resource which is planned to have a new instance added that triggers this issue.

Workaround

I found a workaround for this issue while trying to fix where this came up in our TF code. Moving templatefile to a local and referencing that instead seems to fix this issue. E.g.
replace the data "cloudinit_config" "config" in the reproduction code with

locals {
  config_content = {for k,v in var.test_map: k => templatefile("${path.module}/template.tftpl", {
      value = null_resource.n[k].id
    })}
}
data "cloudinit_config" "config" {
  for_each = var.test_map

  part {
    content = local.config_content[each.key]
  }
}

With this change, step (3) of the reproduction only shows the expected since resource add and not a change to config_id.

Code of Conduct

• I agree to follow this project's Code of Conduct