TERRA-74 ⁃ Dynamic blocks not working with astra_table resource?

[bigjew92]

Hi there,

Please provide the following details with your issue report.

Terraform Version

Terraform v1.1.9
Provider v2.1.4

Affected Resource(s)

Please list the resources as a list, for example:

• astra_table

If this issue appears to affect multiple resources, it may be an issue with Terraform's core, so please mention this.

Terraform Configuration Files

module "testing" {
  source = "../"

  table              = "test"
  keyspace           = "test"
  database_id        = "12345"
  region             = "us-east-1"
  clustering_columns = "a:b"
  partition_keys     = "c:d"

  column_definitions = [
    { name = "astra-dev", static = false, typeDefinition= "text" },
  ]
}

#module resource
resource "astra_table" "table" {
  table              = var.table
  keyspace           = var.keyspace
  database_id        = var.database_id
  region             = var.region
  clustering_columns = var.clustering_columns
  partition_keys     = var.partition_keys
  dynamic "column_definitions" {
    for_each = var.column_definitions
    content {
      name           = column_definitions.value.name
      static         = column_definitions.value.static
      typeDefinition = column_definitions.value.typeDefinition
    }
  }
}

Debug Output

terraform plan
╷
│ Error: Missing required argument
│ 
│   on ../main.tf line 10, in resource "astra_table" "table":
│   10: resource "astra_table" "table" {
│ 
│ The argument "column_definitions" is required, but no definition was found.
╵
╷
│ Error: Unsupported block type
│ 
│   on ../main.tf line 17, in resource "astra_table" "table":
│   17:   dynamic "column_definitions" {
│ 
│ Blocks of type "column_definitions" are not expected here.

Expected Behavior

I am expecting to be able to use dynamic blocks to add multiple column_definitions to each of my tables.

Actual Behavior

The plan fails when trying to use dynamic blocks.

References

Are there any other GitHub issues (open or closed) or Pull Requests that should be linked here? For example:

• TERRA-45 ⁃ astra_access_list only creates the first address #53 might be related? Not sure what was fixed there, but it has to do with dynamic blocks.

┆Issue is synchronized with this Jira Task by Unito
┆friendlyId: TERRA-74
┆priority: Major

[emerkle826]

@bigjew92 Thanks for reporting the issue. I'll take a look today and get back to you.

[emerkle826]

@bigjew92 Sorry for the delay, I've been distracted on some other items. I'll try to get to this in the next day or two.

[emerkle826]

@bigjew92 I don't think you can use an dynamic block in this situation. column_definitions in the table resource is defined as a list, not an object/block type. That's why you get Blocks of type "column_definitions" are not expected here.

[bigjew92]

Gotcha. Thanks @emerkle826. My goal was to be able to pass these to a module, since there can be N number of column_definitions, I was hoping to use dynamic blocks to have them defined (as above) and be able to pass N number of blocks of definitions as I want, if that makes sense.

Doesn't sound like that's possible with the way it is currently implemented, but would definitely be desired.

[emerkle826]

@bigjew92
You might be able to try something like this:

resource "astra_table" "table" {
  table              = var.table
  keyspace           = var.keyspace
  database_id        = var.database_id
  region             = var.region
  clustering_columns = var.clustering_columns
  partition_keys     = var.partition_keys
  column_definitions = [
    dynamic "column_defs" {
      for_each = var.column_definitions
      content {
        name           = column_defs.value.name
        static         = column_defs.value.static
        typeDefinition = column_defs.value.typeDefinition
      }]
  }
}

But I have not tried it. Each column definition is a map of string values, which might work like above, but again I haven't tested it. I'm traveling at the moment, so I can try it when I get where I'm going if you haven't by then

[emerkle826]

If it doesn't work, I'll try to see if I can come up with something that is allowed and gets you closer to what you need.

[bigjew92]

Thanks for the suggestion @emerkle826. We tried that and it wasn't working. The dynamic block constructs repeatable blocks within a resource. So this would make multiple column_defs inside of column_definitions, which the resource is not expecting.

Would appreciate any ideas on how to get to the goal. Will keep trying things and let you know if we come up with something as well.

[milalima]

Hi @emerkle826, I tried another way:

`resource "astra_table" "table" {
table = var.table
keyspace = var.keyspace
database_id = var.database_id
region = var.region
clustering_columns = var.clustering_columns
partition_keys = var.partition_keys

for_each = var.column_definitions
column_definitions = [
{
Name = each.value.Name
Static = each.value.Static
TypeDefinition = each.value.TypeDefinition
}
]

}`

And this is the error that I am getting:

`Error: Plugin did not respond
│
│ with module.brooklyn_dev.astra_table.table["table2"],
│ on ../main.tf line 10, in resource "astra_table" "table":
│ 10: resource "astra_table" "table" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│ with module.brooklyn_dev.astra_table.table["table3"],
│ on ../main.tf line 10, in resource "astra_table" "table":
│ 10: resource "astra_table" "table" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│ with module.brooklyn_dev.astra_table.table["table1"],
│ on ../main.tf line 10, in resource "astra_table" "table":
│ 10: resource "astra_table" "table" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may contain more details.
╵

Stack trace from the terraform-provider-astra_v2.1.4 plugin:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x80 pc=0x1011b09b0]

goroutine 15 [running]:
github.com/datastax/terraform-provider-astra/v2/internal/provider.newRestClient.func1({0x101476468, 0x140007a2060}, 0x0, {0x101464e60, 0x1400039a810})
github.com/datastax/terraform-provider-astra/v2/internal/provider/provider.go:164 +0x20
github.com/hashicorp/go-retryablehttp.(*Client).Do(0x1400055c070, 0x140007a4870)
github.com/hashicorp/go-retryablehttp@v0.7.1/client.go:624 +0x614
github.com/hashicorp/go-retryablehttp.(*RoundTripper).RoundTrip(0x14000534000, 0x140004d0700)
github.com/hashicorp/go-retryablehttp@v0.7.1/roundtripper.go:44 +0x84
net/http.send(0x140004d0700, {0x101462c60, 0x14000534000}, {0x0, 0x0, 0x0})
net/http/client.go:252 +0x594
net/http.(*Client).send(0x1400057a150, 0x140004d0700, {0x0, 0x0, 0x0})
net/http/client.go:176 +0xa4
net/http.(*Client).do(0x1400057a150, 0x140004d0700)
net/http/client.go:725 +0x858
net/http.(*Client).Do(0x1400057a150, 0x140004d0700)
net/http/client.go:593 +0x30
github.com/datastax/astra-client-go/v2/astra-rest-api.(*Client).CreateTable(0x140006060c0, {0x101476468, 0x140007a2060}, {0x14000536578, 0x8}, 0x1400060c440, {{0x1400060a270, 0x1, 0x1}, 0x14000610109, ...}, ...)
github.com/datastax/astra-client-go/v2@v2.2.40/astra-rest-api/astra-rest-api.gen.go:1389 +0x2f4
github.com/datastax/terraform-provider-astra/v2/internal/provider.resourceTableCreate.func1()
github.com/datastax/terraform-provider-astra/v2/internal/provider/resource_table.go:182 +0x200
github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource.RetryContext.func1()
github.com/hashicorp/terraform-plugin-sdk/v2@v2.18.0/helper/resource/wait.go:27 +0x50
github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource.(*StateChangeConf).WaitForStateContext.func1(0x1400054a0c0, 0x1400055c150, 0x14000572060, 0x1400054c140, 0x14000536010, 0x14000536008)
github.com/hashicorp/terraform-plugin-sdk/v2@v2.18.0/helper/resource/state.go:110 +0x178
created by github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource.(*StateChangeConf).WaitForStateContext
github.com/hashicorp/terraform-plugin-sdk/v2@v2.18.0/helper/resource/state.go:83 +0x174

Error: The terraform-provider-astra_v2.1.4 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.`

Do you know what might happen here?

[emerkle826]

@milalima Sorry I haven't replied in a while. I'm taking a look now

[emerkle826]

@bigjew92 @milalima Apologies for the delay. I might have a solution, though it really only moves the column definitions around. But here goes.

In a local module called "testing", you could define a variables.tf like this

variable "column_definitions" {
  type = list(map(string))
    
  default = [
    {
      Name: "a"
      Static: false
      TypeDefinition: "text"
    },
    {
      Name: "b"
      Static: false
      TypeDefinition: "text"
    },
    {
      Name: "c"
      Static: false
      TypeDefinition: "text"
    },
    {
      Name: "d"
      Static: false
      TypeDefinition: "text"
    },
    {
      Name: "e"
      Static: false
      TypeDefinition: "text"
    },
    {
      Name: "f"
      Static: false
      TypeDefinition: "text"
    }
  ]
}

Along side that file, declare an outputs.tf like this:

output "column_defs" {
  value = var.column_definitions
}

Now, in the main module, you can pull in the "testing" module and reference the column_defs output like this:

module "testing" {
  source = "./modules/testing"
}

resource "astra_table" "table" {
  table              = "table1"
  keyspace           = "somekeyspace"
  database_id        = "somedbid"
  region             = "us-east-1"
  clustering_columns = "a:b"
  partition_keys     = "c:d"

  column_definitions = module.testing.column_defs
}

I'm not sure if this really addresses the desire, as it doesn't use any Terraform loops (for or for each) to create the column definitions. But it does get them out of the main module and you can at least keep the column definitions contained within the variables of the child module. And in looking back as I type, the clustering_columns and partition_keys probably ought to be defined in the same place as the column_definitions as they are dependent.

Let me know if this is sufficient. Outside of this solution, I'm afraid the resource structure for a table would have to change to support a more dynamic approach.

[emerkle826]

@milalima To address you question, I'm not sure what is going on from the snippet you provided. However, if something like below would work:

resource "astra_table" "table" {
  table = var.table
  keyspace = var.keyspace
  database_id = var.database_id
  region = var.region
  clustering_columns = var.clustering_columns
  partition_keys = var.partition_keys

  for_each = var.column_definitions
    column_definitions = [
    {
      Name = each.value.Name
      Static = each.value.Static
      TypeDefinition = each.value.TypeDefinition
    }
  ]
}
}

It would create a table for each column_definition in column_definitions, but all would have the same table name, which is likely not what you are going for.

[bigjew92]

Thanks @emerkle826. I will take a look at the first option of defining it as a variable within the module like you've done. It might just be easier to keep it as a flat resource for now/going forward. The goal was to hopefully make it easier for developers that we support to be able to call the module, slightly easier than just creating resources. For now, I think we are okay. If in the future the table resource changes, that would be huge for our use case. If that can be submitted as a feature request, would really appreciate that.

Thanks again for all your help!