Skip to content

Support OIDC / federated ID token refresh during long-running Terraform apply (like azurerm provider) #2834

@geremy42

Description

@geremy42

Description

When using the Kubernetes and Helm Terraform providers with exec-based authentication (e.g. kubelogin in workload identity / OIDC mode), authentication fails during long-running Terraform applies because the OIDC federated token (e.g. from AZURE_FEDERATED_TOKEN_FILE) expires and is never refreshed for the duration of the apply. The azurerm provider (and azurerm backend) already support ID token refresh in this scenario; the same behavior is needed for providers that rely on kubelogin (or similar exec plugins) reading a federated token file.

Environment: Terraform 1.14.x, hashicorp/kubernetes (2.38.x), hashicorp/helm (3.1.x), kubelogin for AKS, Azure DevOps with Workload Identity Federation (OIDC). The pipeline writes the OIDC ID token to a file and sets AZURE_FEDERATED_TOKEN_FILE; kubelogin is used as the exec plugin and reads that file.

Current behavior: Before apply, the pipeline writes the OIDC token once. If apply runs longer than the token lifetime (~10 minutes), the federated token file is never updated. When Terraform later invokes the exec plugin (e.g. for a Kubernetes or Helm resource), kubelogin reads the expired token and fails:

Error: ... failed to create kubernetes rest client for update of resource: Get "https://<cluster>.hcp.<region>.azmk8s.io:443/api?timeout=32s": getting credentials: exec: executable kubelogin failed with exit code 1

Desired behavior (aligned with azurerm): Support OIDC / federated ID token refresh so that when the exec plugin (e.g. kubelogin) is invoked and the current federated token is expired, the provider (or the exec plugin) can obtain a new ID token (e.g. via the same mechanism as Terraform’s Azure DevOps OIDC refresh) and continue. Long-running applies using Kubernetes/Helm with OIDC (kubelogin) should keep working after the initial token expires.

Why it matters: Many users run Terraform in Azure DevOps (or GitHub Actions) with OIDC / workload identity and no long-lived secrets. Applies that exceed the token lifetime fail partway through, forcing workarounds (phased apply, reordering) instead of first-class token refresh like azurerm.

Potential Terraform Configuration

# Kubernetes and Helm providers using kubelogin with workload identity (OIDC).
# AZURE_FEDERATED_TOKEN_FILE is set by the pipeline and points to the OIDC token file.
# When apply runs longer than the token lifetime (~10 min), exec fails because the token is never refreshed.

provider "kubernetes" {
  host                   = azurerm_kubernetes_cluster.this.kube_config[0].host
  cluster_ca_certificate = base64decode(azurerm_kubernetes_cluster.this.kube_config[0].cluster_ca_certificate)

  exec {
    api_version = "client.authentication.k8s.io/v1beta1"
    command     = "kubelogin"
    args = [
      "get-token",
      "--login", "workloadidentity",
      "--environment", "AzurePublicCloud",
      "--server-id", "<azure-ad-server-app-id>"
    ]
    env = {
      AZURE_CLIENT_ID      = data.azurerm_client_config.current.client_id
      AZURE_TENANT_ID      = data.azurerm_client_config.current.tenant_id
      AZURE_AUTHORITY_HOST = "https://login.microsoftonline.com/"
      # AZURE_FEDERATED_TOKEN_FILE is set by CI (e.g. Azure DevOps); token expires ~10 min, never refreshed during apply
    }
  }
}

References

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions