[Bug]: Cannot create hcloud_storage_box_subaccounts

[jdelic]

What happened?

Terraform will try to create storage box subaccounts in parallel. Creating the first one fails for unknown reasons (I can create it on the web interface), but every other one fails because the resource is locked.

[...]
Error: API request failed
│ 
│   with hcloud_storage_box_subaccount.serveraccounts["lb1.*****"],
│   on main.tf line 262, in resource "hcloud_storage_box_subaccount" "serveraccounts":
│  262: resource "hcloud_storage_box_subaccount" "serveraccounts" {
│ 
│ An unexpected error was encountered during an API request.
│ 
│ Action is locked
│ 
│ Error code: locked
│ Status code: 423
│ 
╵
╷
│ Error: Action failed
│ 
│   with hcloud_storage_box_subaccount.serveraccounts["apps1.*****"],
│   on main.tf line 262, in resource "hcloud_storage_box_subaccount" "serveraccounts":
│  262: resource "hcloud_storage_box_subaccount" "serveraccounts" {
│ 
│ An API action for the resource failed.
│ 
│ creating subaccount failed
│ 
│ Error code: action_failed
│ Command: create_subaccount
│ ID: 598022669323684
│ Resources: storage_box: 500400, storage_box_subaccount: 159164

What did you expect to happen?

I expected around 5 subaccounts to be created.

Please provide a minimal working example

Here is the code snippet that creates this error:

locals {
    server_config = {
        "db.local" = {
            server_type = "cx22"
        }
        "test.local" = {
            server_type = "cx22"
        }
        "test2.local" = {
            server_type = "cx32"
        }
    }
}
        
resource "random_password" "storage_box_root" {
    length = 32
    special = true
    override_special = "$%+-#"
    min_special = 1
    upper = true
    min_upper = 1
    lower = true
    min_lower = 1
    min_numeric = 1
}

resource "hcloud_storage_box" "backup-box" {
    name = "backup-box"
    storage_box_type = "bx11"
    location = "hel1"

    password = random_password.storage_box_root.result

    access_settings = {
        ssh_enabled = true
    }

    ssh_keys = [hcloud_ssh_key.ssh1]
    delete_protection = false

    depends_on = [hcloud_ssh_key.ssh1]
}


resource "random_password" "backup_accounts" {
    for_each = local.server_config

    length = 32
    special = true
    override_special = "$%+-#"
    min_special = 1
    upper = true
    min_upper = 1
    lower = true
    min_lower = 1
    min_numeric = 1
}

resource "hcloud_storage_box_subaccount" "serveraccounts" {
    for_each = local.server_config

    storage_box_id = hcloud_storage_box.backup-box.id
    home_directory = "/server/${each.key}/"
    password = random_password.backup_accounts[each.key].result

    access_settings = {
        ssh_enabled = true
    }

    depends_on = [hcloud_storage_box.backup-box]
}

[apricote]

Hey @jdelic,

Creating the first one fails for unknown reasons (I can create it on the web interface)

Could you open a support ticket if you still encounter this issue?

but every other one fails because the resource is locked.

we already saw this during development and added more retries for creating new subaccounts. Unfortunately there is a limit for how much we can do in the Terraform provider.

We are looking into adding support for the actions endpoints for individual resources, so we can "wait" for other resources to finish instead of blindly retrying every resource until they work.

Until that is implemented, I will increase the number of retries.

[apricote]

Ah, I see. I originally added the retry in #1284, but accidentally reverted the change in a rebase of #1166.

[jooola]

This should be fixed in v1.58.0, closed as resolved.

[jdelic]

@jooola
Unfortunately, this doesn't appear to be fixed. Now all subaccount creation attempts fail in my testing with v1.58.0.

❯ terraform version
Terraform v1.14.3
on linux_amd64
+ provider registry.terraform.io/hashicorp/random v3.7.2
+ provider registry.terraform.io/hetznercloud/hcloud v1.58.0

Error: Action failed
│
│   with hcloud_storage_box_subaccount.saltmaster,
│   on main.tf line 228, in resource "hcloud_storage_box_subaccount" "saltmaster":
│  228: resource "hcloud_storage_box_subaccount" "saltmaster" {
│
│ An API action for the resource failed.
│
│ creating subaccount failed
│
│ Error code: action_failed
│ Command: create_subaccount
│ ID: 605263984393752
│ Resources: storage_box: 514288, storage_box_subaccount: 176701
╵
╷
│ Error: Action failed
│
│   with hcloud_storage_box_subaccount.serveraccounts["mail.*****"],
│   on main.tf line 284, in resource "hcloud_storage_box_subaccount" "serveraccounts":
│  284: resource "hcloud_storage_box_subaccount" "serveraccounts" {
│
│ An API action for the resource failed.
│
│ creating subaccount failed
│
│ Error code: action_failed
│ Command: create_subaccount
│ ID: 605263984393735
│ Resources: storage_box: 514288, storage_box_subaccount: 176696
╵
╷
│ Error: Action failed
│
│   with hcloud_storage_box_subaccount.serveraccounts["db.*****"],
│   on main.tf line 284, in resource "hcloud_storage_box_subaccount" "serveraccounts":
│  284: resource "hcloud_storage_box_subaccount" "serveraccounts" {
│
│ An API action for the resource failed.
│
│ creating subaccount failed
│
│ Error code: action_failed
│ Command: create_subaccount
│ ID: 605263984393746
│ Resources: storage_box: 514288, storage_box_subaccount: 176698
╵
[...]

[jooola]

@jdelic Could you please open a support ticket, so the support team can investigate more into your problem?

[jdelic]

Filed a ticket under number 2026011703009913.

Here is also a fully self-contained example config that reproduces the error:

terraform {
    required_providers {
        hcloud = {
            source  = "hetznercloud/hcloud"
            version = "> 1.0"
        }
        random = {
            source  = "hashicorp/random"
            version = "> 3.6.0"
        }
    }
}

variable "hcloud_token" {
    sensitive = true
}

provider "hcloud" {
    token = var.hcloud_token
}

locals {
    server_config = {
        "db.local" = {
            server_type = "cx22"
        }
        "test.local" = {
            server_type = "cx22"
        }
        "test2.local" = {
            server_type = "cx32"
        }
    }
}
        
resource "random_password" "storage_box_root" {
    length = 32
    special = true
    override_special = "$%+-#"
    min_special = 1
    upper = true
    min_upper = 1
    lower = true
    min_lower = 1
    min_numeric = 1
}

resource "hcloud_storage_box" "backup-box" {
    name = "backup-box"
    storage_box_type = "bx11"
    location = "hel1"

    password = random_password.storage_box_root.result

    access_settings = {
        ssh_enabled = true
    }

    delete_protection = false
}


resource "random_password" "backup_accounts" {
    for_each = local.server_config

    length = 32
    special = true
    override_special = "$%+-#"
    min_special = 1
    upper = true
    min_upper = 1
    lower = true
    min_lower = 1
    min_numeric = 1
}

resource "hcloud_storage_box_subaccount" "serveraccounts" {
    for_each = local.server_config

    storage_box_id = hcloud_storage_box.backup-box.id
    home_directory = "/server/${each.key}/"
    password = random_password.backup_accounts[each.key].result

    access_settings = {
        ssh_enabled = true
    }

    depends_on = [hcloud_storage_box.backup-box]
}

To run use:

export TF_VAR_hcloud_token="[your api token]"
terraform init
terraform apply

[jdelic]

Support replied to me that home_directory may not start with a slash. That this fails with a nondescript API error message is ... unfortunate, but without the leading /, subaccounts can be created.