[ISSUE] Issue with `databricks_catalog` resource - name update reports success without renaming

[pgondi13]

Configuration

variable "catalog_name" {
  type = string
}

variable "workspace_id" {
  type = number
}

variable "catalog_storage_root" {
  type = string
}

variable "principal" {
  type = string
}

resource "databricks_catalog" "example" {
  name           = var.catalog_name
  storage_root   = var.catalog_storage_root
  isolation_mode = "ISOLATED"
}

resource "databricks_catalog_workspace_binding" "example" {
  securable_name = databricks_catalog.example.name
  workspace_id   = var.workspace_id
}

resource "databricks_grant" "example" {
  catalog    = databricks_catalog.example.name
  principal  = var.principal
  privileges = [
    "USE_CATALOG",
    "USE_SCHEMA",
    "SELECT",
  ]
}

The first apply uses:

catalog_name = "example_catalog"

The second apply changes it to:

catalog_name = "example_catalog_rollback"

Expected Behavior

One of the following should happen:

1. If catalog renaming is supported, the provider should rename the catalog using the Catalogs Update API, update the Terraform resource ID/state, and then update dependent resources to use the new name.
2. If catalog renaming is not supported, the provider should reject the change during planning or clearly mark the resource for replacement.

The provider should not report that the catalog modification completed successfully when the catalog was not renamed. A failed rename should also not leave grants deleted from the existing catalog.

Actual Behavior

Terraform planned the databricks_catalog.name change as an in-place update.

During apply, Terraform first destroyed databricks_grant resources whose catalog field was changing. The provider then reported that the catalog modification completed:

databricks_catalog.example: Modifying...
databricks_catalog.example: Modifications complete

Terraform subsequently failed with Provider produced inconsistent final plan for resources that referenced the catalog name.

The workspace binding error was equivalent to:

Error: Provider produced inconsistent final plan

When expanding the plan for
databricks_catalog_workspace_binding.example, the provider produced an
invalid new value for .securable_name: the planned value was
"example_catalog_rollback", but the provider returned
"example_catalog".

The grant resources produced the same type of error for their catalog attributes.

After the failed apply, I checked Unity Catalog directly:

• example_catalog still existed.
• example_catalog_rollback did not exist.
• The grants destroyed before the catalog update remained deleted.
• The catalog rename therefore did not occur, despite the provider reporting that its modification completed.

This resulted in a partial and destructive apply: the original catalog remained under its original name but no longer had all of its previously managed grants.

Steps to Reproduce

1. Create a Unity Catalog catalog with catalog_name = "example_catalog".
2. Create a databricks_catalog_workspace_binding and one or more databricks_grant resources that reference databricks_catalog.example.name.
3. Run terraform apply.
4. Change catalog_name to "example_catalog_rollback".
5. Run terraform plan.
6. Confirm that the catalog name change is planned as an in-place update and the dependent grants are planned for replacement.
7. Run terraform apply.
8. Observe that the provider reports the catalog modification as complete.
9. Observe the inconsistent-final-plan errors for the binding and grants.
10. Query Unity Catalog directly and confirm that the original catalog still exists and the renamed catalog does not.
11. Observe that grants destroyed earlier in the apply have not been restored.

Terraform and provider versions

Terraform version: 1.13.5

The Databricks provider version used in the failed apply was:

registry.terraform.io/databricks/databricks v1.111.0

Is it a regression?

Unknown. We have not successfully renamed a Terraform-managed Unity Catalog catalog with an earlier provider version.

We have not reproduced this against the latest provider yet. However, v1.111.0 accepts the name change as an in-place update and reports that it completed successfully, even though the catalog is not renamed.

Debug Output

A sanitized excerpt from the apply:

databricks_grant.example: Destroying...
databricks_grant.example: Destruction complete

databricks_catalog.example: Modifying...
databricks_catalog.example: Modifications complete

Error: Provider produced inconsistent final plan

When expanding the plan for
databricks_catalog_workspace_binding.example, the provider produced an
invalid new value for .securable_name: the planned value was
"example_catalog_rollback", but the provider returned
"example_catalog".

Error: Provider produced inconsistent final plan

When expanding the plan for databricks_grant.example, the provider
produced an invalid new value for .catalog: the planned value was
"example_catalog_rollback", but the provider returned
"example_catalog".

A full sanitized debug log can be provided if needed. The original log contains private workspace, storage, catalog, and principal identifiers, so I did not want to include it.

Important Factoids

• I found https://community.databricks.com/t5/administration-architecture/why-catalog-api-does-not-include-the-catalog-id-in-the-response/m-p/119055 which says it's impossible to rename a catalog through Terraform.

Would you like to implement a fix?

No, not currently. Our workaround is to use the CLI, but it requires messy Terraform state surgery to reconcile things so it's not ideal.