Unable to create business units in Commercetools with custom fields

[nakhodkin]

🐞 Version information

You can reproduce the issue using either Terraform or OpenTofu

Terraform

• Terraform CLI: v1.11.3 (darwin_arm64)
• Provider: registry.terraform.io/labd/commercetools v1.19.3

OpenTofu

• OpenTofu CLI: v1.9.0 (darwin_arm64)
• Provider: registry.opentofu.org/labd/commercetools v1.19.3

🧩 Describe the Bug

Applying a configuration that provisions multiple commercetools_business_unit_company resources with optional custom fields results in an error or unexpected behaviour. Specifically, not all custom fields are applied correctly when using conditional logic (? : null) for optional fields of type Reference.

🔁 Steps to Reproduce

1. Export environment variables

export CTP_CLIENT_ID=...
export CTP_CLIENT_SECRET=...
export CTP_PROJECT_KEY=...
export CTP_SCOPES=...
export CTP_API_URL=https://api.europe-west1.gcp.commercetools.com
export CTP_AUTH_URL=https://auth.europe-west1.gcp.commercetools.com

2. Create the following Terraform configuration files

# main.tf
terraform {
  required_providers {
    commercetools = {
      source  = "labd/commercetools"
      version = "1.19.3"
    }
  }
}

resource "commercetools_type" "business-unit-custom-fields" {
  key = "business-unit-custom-fields"

  name = {
    en = "Custom Fields"
  }

  description = {
    en = "Business Unit Custom Fields"
  }

  resource_type_ids = ["business-unit"]

  field {
    name     = "codaAccountNumber"
    required = false
    label    = { en = "My property" }
    type     = { name = "String" }
  }

  field {
    name     = "customerGroup"
    required = false
    label    = { en = "A customer group associated with the business unit" }
    type     = {
      name              = "Reference"
      reference_type_id = "customer-group"
    }
  }
}

resource "commercetools_business_unit_company" "business_units" {
  for_each = { for index, unit in var.business_units : unit.key => unit }

  key  = each.key
  name = each.value.name

  custom {
    type_id = commercetools_type.business-unit-custom-fields.id

    fields = {
      codaAccountNumber = each.value.fields.codaAccountNumber
      customerGroup = each.value.fields.customerGroup != null ? jsonencode({
        id     = each.value.fields.customerGroup
        typeId = "customer-group"
      }) : null
    }
  }
}

# variables.tf
variable "business_units" {
  type = set(object({
    key  = string
    name = string
    fields = optional(object({
      codaAccountNumber = optional(string, null)
      customerGroup = optional(string)
    }), {})
  }))
  description = "List of business units to create in Commercetools"
}

# variables.tfvars
business_units = [
  {
    key = "business-a"
    name = "business as usual A",
  },
  {
    key = "business-b"
    name = "business as usual B",
    fields = {
      codaAccountNumber = "My value 1"
    }
  },
  {
    key = "business-c"
    name = "business as usual C",
    fields = {
      customerGroup = "98d74390-b6ad-4363-83c5-effb6c9e49f7"
    }
  },
  {
    key = "business-d"
    name = "business as usual D",
    fields = {
      codaAccountNumber = "My value 2"
      customerGroup = "98d74390-b6ad-4363-83c5-effb6c9e49f7"
    }
  }
]

3. Initialize Terraform/OpenTofu

$ terraform init

4. Run a plan

$ terraform plan -var-file=variables.tfvars

5. Apply the plan:

terraform apply -var-file=variables.tfvars

✅ Expected behaviour

All four business units should be created in Commercetools with the specified key, name, and custom fields (if provided). Optional custom fields should be omitted cleanly when null.

📸 Screenshots

📎 Additional Context

This issue may be related to how the provider handles conditional serialization of Reference types with null values. Please let me know if a workaround exists or if additional logs would help.

[demeyerthom]

Hi @nakhodkin not an issue with the provider I think, but the terraform. It did not work on first try, so I edited it a bit to be sure. This works:

resource "commercetools_customer_group" "standard" {
  key  = "my-customer-group-key"
  name = "Standard Customer Group"
}

locals {
  business_units = [
    {
      key  = "business-a"
      name = "business as usual A"
      fields = {
        codaAccountNumber = "My value 2"
      }

    },
    {
      key  = "business-b"
      name = "business as usual B",
      fields = {
        codaAccountNumber = "My value 1"
      }
    },
    {
      key  = "business-c"
      name = "business as usual C",
      fields = {
        customerGroup = jsonencode({
          id     = commercetools_customer_group.standard.id
          typeId = "customer-group"
        })
      }
    },
    {
      key  = "business-d"
      name = "business as usual D",
      fields = {
        codaAccountNumber = "My value 2"
        customerGroup = jsonencode({
          id     = commercetools_customer_group.standard.id
          typeId = "customer-group"
        })
      }
    }
  ]
}


resource "commercetools_type" "business-unit-custom-fields" {
  key = "business-unit-custom-fields"

  name = {
    en = "Custom Fields"
  }

  description = {
    en = "Business Unit Custom Fields"
  }

  resource_type_ids = ["business-unit"]

  field {
    name     = "codaAccountNumber"
    required = false
    label = { en = "My property" }
    type { name = "String" }
  }

  field {
    name     = "customerGroup"
    required = false
    label = { en = "A customer group associated with the business unit" }
    type {
      name              = "Reference"
      reference_type_id = "customer-group"
    }
  }
}

resource "commercetools_business_unit_company" "business_units" {
  for_each = {for index, unit in local.business_units : unit.key => unit}

  key  = each.key
  name = each.value.name

  custom {
    type_id = commercetools_type.business-unit-custom-fields.id
    fields  = each.value.fields
  }
}

Can you provide me the terraform that caused the errors?

[nakhodkin]

Hello @demeyerthom

Thank you for your reply!
The goal was to define input variables within a .tfvars file:

# variables.tfvars
business_units = [
  {
    key = "business-a"
    name = "business as usual A",
  },
  {
    key = "business-b"
    name = "business as usual B",
    fields = {
      codaAccountNumber = "My value 1"
    }
  },
  {
    key = "business-c"
    name = "business as usual C",
    fields = {
      customerGroup = "98d74390-b6ad-4363-83c5-effb6c9e49f7"
    }
  },
  {
    key = "business-d"
    name = "business as usual D",
    fields = {
      codaAccountNumber = "My value 2"
      customerGroup = "98d74390-b6ad-4363-83c5-effb6c9e49f7"
    }
  }
]

Without graceful handling of null-s for optional fields by the provider, to make it work, clients need to do something like this:

locals {
  units = [ for index, unit in toset(var.business_units): {
    key = unit.key
    name = unit.name
    fields = { for i, f in ({ for index, field in unit.fields: index => field if field != null  }) : i => (i == "customerGroup") ? jsonencode({
      id = f
      typeId = "customer-group"
    }) : f}
  } ]
}

or as you mentioned correctly, it will be a working alternative as well - to create environment-specific module files with variables defined in locals.

Can you provide me the terraform that caused the errors?

Not sure If I get it 😅 In the original message I provided you with the contents of used Terraform files.
If you're talking about the root module, it is simple as:

module "business_unit" {
  source         = "./modules/business_unit"
  business_units = var.business_units
}

variable "business_units" {
  type = set(object({
    key  = string
    name = string
    fields = optional(object({
      codaAccountNumber = optional(string, null)
      customerGroup = optional(string)
    }), {})
  }))
  description = "List of business units to create in Commercetools"
}

[demeyerthom]

Hi @nakhodkin A I get it now. I was slightly thrown off because your previous example had a minor mistake, but now I am seeing the same error.

This has to do with how we treat the custom fields. We always expect the value to be a string (or stringified json) so we can forward it correctly to commercetools. However this mechanism does not deal well with null values. The code around it is a tad complicated, but I will see if I can improve on that to allow for null values in the future