Migrate to v5 when remote state contains incompatible v4 schema

[peterhirn]

Migrating this very simple terraform setup: phi-ag/rvt-app#175

I'm unable to plan/apply because the state obviously contains the old schema.

terraform apply

Running apply in HCP Terraform. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.

Preparing the remote apply...

To view this run in a browser, visit:
https://app.terraform.io/app/phi/rvt-app/runs/run-tLNtwbNfm8Roz1WG

Waiting for the plan to start...

Terraform v1.9.8
on linux_amd64
Initializing plugins and modules...
╷
│ Error: no schema available for cloudflare_record.production while reading state; this is a bug in Terraform and should be reported
│
│
╵
╷
│ Error: no schema available for cloudflare_record.preview while reading state; this is a bug in Terraform and should be reported
│
│
╵
╷
│ Error: Unable to Read Previously Saved State for UpgradeResourceState
│
│   with cloudflare_pages_project.page,
│   on main.tf line 59, in resource "cloudflare_pages_project" "page":
│   59: resource "cloudflare_pages_project" "page" {
│
│ There was an error reading the saved resource state using the current
│ resource schema.
│
│ If this resource state was last refreshed with Terraform CLI 0.11 and
│ earlier, it must be refreshed or applied with an older provider version
│ first. If you manually modified the resource state, you will need to
│ manually modify it to match the current resource schema. Otherwise, please
│ report this to the provider developer:
│
│ AttributeName("build_config"): invalid JSON, expected "{", got "["
╵
Failed generating plan JSON
Exit code: 1

Failed to marshal plan to json: error marshaling prior state: unsupported attribute "domain"
Operation failed: 2 errors occurred:
        * failed running terraform plan (exit 1)
        * failed generating plan JSON: failed running command (exit 1)

Any recommendations how to handle this?

[jacobbednarz]

have you seen the migration guide? if so, which steps did you take?

[105ron]

Hi Jacob, I am having similar issues with the migration, and have been in contact with support opening case 01356987 but my support engineer is on a different time zone so there's delays between replies.

I have installed the grit tool with npm and have had some success with the patterns. It will rename cloudflare_record to cloudflare_dns_record in the *.tf files for me. However, it when I run the grit apply cloudflare_terraform_v5_resource_renames_state I get:

Your working tree currently has untracked changes and Grit will rewrite files in place. Do you want to proceed? yes
Processed 1 files and found 0 matches

I am having similiar issues with the cloudflare_page_rule resource because of the change in blocks to attributes. I run cloudflare_terraform_v5_attribute_renames_state and get the same output I have pasted above. I get similar errors to @peterhirn when I run my plan. I am using the patterns i found here which don't seem to have the patterns I would need to fix these two separate issues with my state file.

[philip-harvey]

It's rather hidden deep in the fine print of the migration instructions, but their proposed solution to this is to delete every old cloudflare object from your state file using terraform (terraform state rm addressofeachobject), and then manually import every new cloudflare object into your state file.
Probably do-able if you just have a few objects, but a total nightmare if you have a lot of object. I'll be staying on v4.x until this migration process is smoother...

[jacobbednarz]

However, it when I run the grit apply cloudflare_terraform_v5_resource_renames_state I get:

@105ron which of the following approaches are you using from https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/guides/migrating-renamed-resources ?

It's rather hidden deep in the fine print of the migration instructions, but their proposed solution to this is to delete every old cloudflare object from your state file using terraform (terraform state rm addressofeachobject), and then manually import every new cloudflare object into your state file.
Probably do-able if you just have a few objects, but a total nightmare if you have a lot of object.

@philip-harvey there are multiple approaches, some of which don't require you to manual do the import. we recently ran the second option (https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/guides/migrating-renamed-resources) on 10k resources and it took care of it within a couple of minutes.

[105ron]

@105ron which of the following approaches are you using from https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/guides/migrating-renamed-resources ?

Hi @jacobbednarz, I am trying to do this command $ grit apply github.com/cloudflare/terraform-provider-cloudflare#cloudflare_terraform_<version>_<what is changing>_state terraform.tfstate after pulling the state to my local machine (It's hosted in an S3 bucket). When I do it (edit: it being this command --> grit apply github.com/cloudflare/terraform-provider-cloudflare#cloudflare_terraform_V5_resource_renames_state terraform.tfstate) though, I get ERROR (code: 200) - pattern definition not found: cloudflare_terraform_V5_resource_renames_state. Try running grit init. I'm not sure why my grit won't run patterns from the repository, but I've put the patterns in a local directory and the resource renaming works for *.tf so I'm sure my grit works. The patterns I've pulled for this is here. I see it analysing the file, but it makes no changes. I don't see a pattern in there for cloudflare_record so perhaps I am on the wrong branch. If you click through to the repository on this link, you get patterns for v4.40 which doesn't seem to fit with the commands on the terraform Cloudflare provider documentation.

[jacobbednarz]

in that grit pattern you've linked, we only run it on the dependencies not the resource itself - https://github.com/cloudflare/terraform-provider-cloudflare/blob/main/.grit/patterns/cloudflare_terraform_v5_resource_renames_state.grit#L70 🤔 had a dig around and couldn't see a reason why this shouldn't be in there so i went ahead and added it in 763d807.

the configuration (HCL) version looks to cover the resource rename so that should cover you on that front already. let me know how you go.

[jacobbednarz]

i'm definitely seeing it running locally now for config + state

$ grit apply github.com/cloudflare/terraform-provider-cloudflare#cloudflare_terraform_v5_resource_renames_state terraform.tfstate --language json
terraform.tfstate
       "resources": [
         {
           "mode": "managed",
    -      "type": "cloudflare_record",
    +      "type": "cloudflare_dns_record",
           "name": "example",
           "provider": "provider[\"registry.terraform.io/cloudflare/cloudflare\"]",
           "instances": [


Processed 1 files and found 1 matches

$ grit apply github.com/cloudflare/terraform-provider-cloudflare#cloudflare_terraform_v5_resource_renames_configuration terraform.tf
terraform.tf
       default = "03d5bff5108807b98f55be582c081909"
     }

    -resource "cloudflare_record" "example" {
    +resource "cloudflare_dns_record" "example" {
       zone_id = var.zone_id
       name = "foo"
       type = "A"


Processed 1 files and found 1 matches

hopefully this gets you moving!

[105ron]

Well, I have some progress in that I can run those GitHub sourced patterns. The configuration one changes my file names correctly. However on a small module with two records I would like to change the state, it is still finding nothing to change. My exact command is grit apply github.com/cloudflare/terraform-provider-cloudflare#cloudflare_terraform_v5_resource_renames_state terraform.tfstate --language json On our main production zone, with > 1000 DNS records I get:

Skipped terraform.tfstate, it is too big.
Processed 1 files and found 0 matches

So it would appear to be finding the state file and analysing it, it appears as though I don't have the right patterns still. I will give the computer a restart over the weekend and try again on Monday.

[jacobbednarz]

good to see some progress there. i ran those patterns on similar sized files and didn't see the error so we must have been just under it. i'll take a look at whether or not there is something in the tool we can use to override it.

if that fails (or you get impatient), you can do a find and replace on the state file given it's pretty straight forward JSON. we use AST for the codemod but it's not strictly required. these two should be enough.

• "cloudflare_record" => "cloudflare_dns_record"
• "cloudflare_record. => "cloudflare_dns_record.

[peterbaumert]

For me if I run
grit apply github.com/cloudflare/terraform-provider-cloudflare#cloudflare_terraform_v5_resource_renames_state terraform.tfstate --language json

on my state file which includes for example

    },
    {
      "module": "module.cloudflare",
      "mode": "managed",
      "type": "cloudflare_record",
      "name": "*****",
      "provider": "provider[\"registry.terraform.io/cloudflare/cloudflare\"]",
      "instances": [
        {
          "schema_version": 3,
          "attributes": {
            "allow_overwrite": false,

It just results in Processed 1 files and found 0 matches

[maheedharTumpudi]

However, it when I run the grit apply cloudflare_terraform_v5_resource_renames_state I get:

@105ron which of the following approaches are you using from https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/guides/migrating-renamed-resources ?

It's rather hidden deep in the fine print of the migration instructions, but their proposed solution to this is to delete every old cloudflare object from your state file using terraform (terraform state rm addressofeachobject), and then manually import every new cloudflare object into your state file.
Probably do-able if you just have a few objects, but a total nightmare if you have a lot of object.

@philip-harvey there are multiple approaches, some of which don't require you to manual do the import. we recently ran the second option (https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/guides/migrating-renamed-resources) on 10k resources and it took care of it within a couple of minutes.

@jacobbednarz Even I followed second approach given in document (https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/guides/migrating-renamed-resources#manually-modifying-the-state-file) where it suggested to run grit apply for rename pattern of state and then do for rename pattern of modules. But after doing these they are just renaming the resource types, but not the structural change. As a result of this when i run terraform plan its throwing errors saying

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Resource instance managed by newer provider version
│
│ The current state of module.wildcardtest2-factset-com-cname-record.cloudflare_dns_record.cname-record["wildcardtest2.factset.com/1/test4"] was created by a newer provider version than is currently selected.
│ Upgrade the cloudflare provider to work with this state.

I had to remove the resource from state file and then import it manually which fixed the issue. But i cannot do this as we have more than 8000 resources of cloudflare_record type where import all resources is not possible.

[105ron]

@105ron which of the following approaches are you using from https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/guides/migrating-renamed-resources ?

I am using the state pull, apply the grit patterns, push state and verify. Though I really haven't got past the grit part yet, because it's not working correctly for me.

if that fails (or you get impatient), you can do a find and replace on the state file given it's pretty straight forward JSON. we use AST for the codemod but it's not strictly required. these two should be enough.

This morning when I run the command it's changing cloudflare_worker_script to cloudflare_workers_script for me in the state file, but skipping cloudlfare_record. I see the cloudflare_record => cloudflare_dns_record is on the main branch pattern on the Cloudflare Terraform provider GitHub so I pulled that down and add it to my local ~/.grit/patterns and have success with it on a medium sized zone - though it only did cloudflare_record migration for me. The issue we have is we have split each zone into modules and they each have a collection of resources. It's the cloudflare_page_rule changing blocks to attributes that prevents me from running a plan. There may be some other resources that are going to have the same issue with the re architecture of the provider and multiple resources changing from blocks to attributes. Currently I can't find a pattern or way to migrate these blocks to attributes within the state file for resources to V5 state file.

[jacobbednarz]

It's the cloudflare_page_rule changing blocks to attributes that prevents me from running a plan.
[...]
There may be some other resources that are going to have the same issue with the re architecture of the provider and multiple resources changing from blocks to attributes.

cloudflare_page_rule is one of the (very limited, < 3) resources that couldn't be automatically migrated in it's entirety due to the way the resources are setup. the other block migrations are already covered in https://github.com/cloudflare/terraform-provider-cloudflare/blob/main/.grit/patterns/cloudflare_terraform_v5_block_to_attribute_configuration.grit. calling github.com/cloudflare/terraform-provider-cloudflare#cloudflare_terraform_v5 should already cover those. the grit patterns for attributes do not modify the state; you'll need to reimport them as you perform the upgrade.

[105ron]

good to see some progress there. i ran those patterns on similar sized files and didn't see the error so we must have been just under it. i'll take a look at whether or not there is something in the tool we can use to override it.

if that fails (or you get impatient), you can do a find and replace on the state file given it's pretty straight forward JSON. we use AST for the codemod but it's not strictly required. these two should be enough.

* `"cloudflare_record"` => `"cloudflare_dns_record"`

* `"cloudflare_record.` => `"cloudflare_dns_record.`

Ok, just in the interest of sharing what I have found today. After deleting my cloudflare_page_rule resources with a view to importing them later, I renamed cloudflare_record in the state file as suggested by @jacobbednarz in this comment. I still had errors, I have found that in my records of type "cloudflare_record in my state file that in addition to changing the type field I needed to remove from instances[0] the attribute "allow_overwrite" and if "data is set to [], change it to null. That then got a basic zone running on the V5 provider for me allowing me to run a plan. I'll try tomorrow on a bigger state file and see if it works.