Proposal: Enable plan checking support when migrating between schema versions

[gdavison]

terraform-plugin-testing version

github.com/hashicorp/terraform-plugin-testing v1.11.0

Use cases

Testing provider enhancements that include schema changes is challenging with the current acceptance testing framework. If there are no schema changes, using ExternalProviders with a previous version and then a PlanOnly step works well.

However, this technique does not work when there are schema changes. Using PlanOnly will display changes that the user will see, but fail either with The non-refresh plan was not empty if there are changes, or a message such as Error retrieving state, there may be dangling resources: exit status 1 Failed to marshal state to json: schema version 0 for aws_s3_bucket_lifecycle_configuration.test in state does not match version 1 from the provider if there are no value changes.

Using ConfigPlanChecks.PreApply seems like it should work in this case. However, it does not show the changes between the existing state and the configured state, but instead the existing state after the state upgrade has been run and the configured state.

Proposal

Two possibilities:

1. If the underlying terraform calls can return the state without applying the state upgrader, the call used by ConfigPlanChecks.PreApply should be updated to use that
2. Otherwise, add a mechanism to collect state values similarly to how statecheck.CompareValue() works

[austinvalle]

Alright, I’m looking at this issue again and I see that the PR has been merged related to it in the AWS provider. I also found the old issue that Kit opened related to this:

• Error with state upgrades after terraform-plugin-testing@v1.6.0 update #237
• Error with state upgrades after terraform-plugin-testing@v1.6.0 update #237 (comment)
• Fix state upgrade acceptance tests cases that check via PlanOnly: true terraform-provider-aws#34742

Test ref: https://github.com/hashicorp/terraform-provider-aws/blob/d6205e532b18d33402745c2d0c5e85231d6c6917/internal/service/s3/bucket_lifecycle_configuration_migrate_test.go#L42

So in step 2, we run ConfigPlanChecks.PreApply , which is run after a terraform plan that refreshes/upgrades the state, which you can then verify that a change occurs in the plan.

Were you wanting to verify what the data is in state before step 2 starts? (after the upgrade state is run) I would’ve thought that you’d want to verify the proposed plan change, which is seems like you were able to do