Ephemeral External Resource

[Bradley-McCallion]

Terraform CLI and Provider Versions

Terraform v1.10.2
on linux_amd64

• provider registry.terraform.io/hashicorp/external v2.3.5
• provider registry.terraform.io/integrations/github v6.6.0

Use Cases or Problem Statement

Sometimes your external data source can contain sensitive information that you wouldn't want to store in a state file. For example,

data "external" "decrypted_secrets" {
  program = [
    "${path.module}/scripts/decrypt-secrets-sops.sh",
    "${secrets_folder}/${var.secret_type}.yaml"
  ]
}

locals {
  decrypted_secrets = sensitive(jsondecode(data.external.decrypted_secrets.result))
  secrets_loop = var.secret_type == "repository" || var.secret_type == "repository-environment" ? {
    for key, val in local.decrypted_secrets :
    key => val if split(".", key)[0] == var.repository_slug
    } : {
    for key, val in local.decrypted_secrets : key => val
  }
  sensitive = true
}

resource "github_actions_secret" "this" {
  for_each = var.secret_type == "repository" ? local.secrets_loop : {}

  repository      = var.repository_name
  secret_name     = split(".", each.key)[1]
  plaintext_value = each.value
}

Here I am running a script that will decrypt a secrets file using sops and then use the result to create github secrets. This provides a nice way to store secrets via code in a relatively safe manner Ideally this result of the external data source would not be stored in the state file for security reasons. Furthermore as external data source will run on every plan operation, there is no real need or use for a cached result to be stored in state.

Of course the utility of this largely depends on the downstream use of the result from the external program being able to be used in an ephemeral value for a downstream resource. For example, here I am then storing the sensitive secret in the plaintext_value property which will currently still be stored in state meaning the secret is exposed anyway. However, having the external program's result (and therefore the locals) at least be ephemeral is a good start. I can then open an issue in the github provider to ask if they can make the plaintext_value ephemeral. In this way, the secret values won't be stored in state at all.

NOTE I am currently getting around this by just encrypting the state file but of course it would be nice to layer this security

Proposal

The provider hashicorp/external supports ephemeral resource "external":

ephemeral "external" "decrypted_secrets" {
  program = [
    "${path.module}/scripts/decrypt-secrets-sops.sh",
    "${local.secrets_folder}/${var.secret_type}.yaml"
  ]
}

How much impact is this issue causing?

Medium

Additional Information

Workaround is to encrypt the state file currently.

Code of Conduct

• I agree to follow this project's Code of Conduct

[busser]

I opened a PR that implements this feature: #442 :)