Clarification: `terraform_remote_state` or `tfe_outputs`

[nairb774]

I recently upgraded the tfe provider to v0.73.0 and noticed the warning to set a TFE_TOKEN env var or the token argument. I'd been using the tfe_outputs datasource because of the recommendation found on https://developer.hashicorp.com/terraform/language/state/remote-state-data. Could you provide updated documentation on how to create a token for this cross-workspace output reading with scoped permissions?

Use-cases

Read outputs from one workspace in another workspace while easily managing permissions. I'd like to be able to preserve the permissions functionality afforded by these settings in the Terraform Cloud UI if possible:

Attempted Solutions

I've been using tfe_outputs successfully for awhile due to the suggestion on https://developer.hashicorp.com/terraform/language/state/remote-state-data.
I could revert back to using terraform_remote_state if that is the best option.

Proposal

Either update the documentation on the best path for reading tfe_outputs or could you provide documentation on how to make use of this existing security feature with the newly suggested TFE_TOKEN path?

[chrisgavin]

We've also started hitting this warning as a result of using tfe_outputs since #1890 was merged.

@christian-doucette do you know if this is expected behavior? It's not clear to me if the warning is intended to apply in this case.

We're not explicitly using a configuration file to pass a token, but this just seems to be the default behavior of the Terraform runner.

HCP Terraform generates a unique per-run API token and provides it to the Terraform worker in the CLI config file

[robertpscully]

The token that is generated for the worker won't have the specific permissions required to read from the Workspace you're trying to read the data from.

The remote state sharing configuration that can be set at the workspace level isn't really part of the standard permissions model that the rest of HCP Terraform follows for allowing access to internal resources.

Allowing one Workspace access to the outputs from another requires providing the "reading" Workspace with a TFE_TOKEN. The owner of this token must have the explicit permission to 'read outputs' in the statefile versions permissions from the "output" Workspace.

Treat this as the same requirement you would have if you were running the Workspace in local mode, and were configuring the tfe provider to read a tfe_output object without the aid of a worker.

The practice that I've followed to achieve this (at a fairly large scale) is

Create and assign a token

This part will generate you a token that can be used in the scope of either one workspace or as part of a multiple projects

• create a tfe_team (called "Workspace Output Readers" or something similar).
  • This team does not need to have any team members
• create an ephemeral tfe_team_token for this team
• create a tfe_variable using the value_wo argument
• set the usage scope for the variable
  • set the "reading" Workspace id as the tfe_variable argument for workspace_id OR
  • create a tfe_variable_set for and set this as the variable_set_id if you want to use this in multiple places

Create and assign token RBAC

The team that owns the token can now be given access to the workspace you need to read the outputs from.
If you want all workspaces in a project to be able to read each others outputs this can also be set with a project level access.

• decide the permission scope for the team
  • create a tfe_team_access for the team and each "output" workspace you want to read the output from
  • create a tfe_team_project_access for the team and each project you want to read the output from
• set the actual role using the permissions block in the access resource
  • the minimum permission required is to set the value of the state_versions argument to read-outputs in the permission block

Least privilege

Creating the above resources with Terraform requires a token with significant privilege. I've used the organization token to do this when deploying teams for this purpose.

Creating one team for doing all the output reading, and assigning it everywhere is tempting, but you should scale this to match the least privilege for what workspaces need access to other workspaces for output access.

Hope this helps

[chrisgavin]

Thanks for the information @robertpscully. I'm not sure I understand what you mean when you say:

The token that is generated for the worker won't have the specific permissions required to read from the Workspace you're trying to read the data from.

Isn't "Remote state sharing" meant to give the token that access? Or is it that this feature is deprecated and the warning is telling people to stop using it?

[robertpscully]

I've just reproduced the deprecation warning in a test to understand, and it looks like the latter ,that it's something that has become deprecated and you can't rely on the agent token for future versions.

I came across an issue when using a private agent pool and an explicit TFE_TOKEN I was setting on a workspace when enabling hyok, with the implicit token breaking the explicit token, and I'm now wondering if the fix into the private agents hasn't caused a general review of its usage.

#1908

---

It's a very HCP Terraform specific piece of configuration for enabling this remote state sharing.

Relying upon the workers ability to read the invoking workspace's statefile, workspace variables etc. to access other HCPT resources feels like an abuse of this.

If you were trying to share state or outputs from a workflow that is using HCPT as a backend, with one that is using S3 for example, the only way to achieve this is to use an explicit token for the tfe provider, and I suspect this will be the only way in the future.