Unable to mock data source attribute of list type using `override_data`, suspected due to missing type conversion

[sanderginn]

Terraform Version

1.14.0

Terraform Configuration Files

main.tf

terraform {
  required_providers {
    databricks = {
      source  = "databricks/databricks"
      version = "1.97.0"
    }
  }

  required_version = ">= 1.14.0"
}

data "databricks_external_location" "test" {
  name = "test"
}

test.tftest.hcl

mock_provider "databricks" {}

run "nothing_mocked_failure_shows_generated_id" {
  override_data {
    target = data.databricks_external_location.test
  }

  command = plan
  
  assert {
    condition = data.databricks_external_location.test.id == "expecting failure, not mocked"
    error_message = "See diff for generated ID"
  }
}

run "id_mocked_successfully" {
  override_data {
    target = data.databricks_external_location.test
    values = {
      id = "with-id-mocked"
    }
  }
  
  command = plan

  assert {
    condition = data.databricks_external_location.test.id == "with-id-mocked"
    error_message = "Expected the external location ID to be 'with-id-mocked'"
  }
}

run "external_location_url_mocked_unsuccessfully" {
  override_data {
    target = data.databricks_external_location.test
    values = {
      external_location_info = [{ url = "abfss://container@foo.bar/" }]
    }
  }

  command = plan

  assert {
    condition = data.databricks_external_location.test.external_location_info[0].url == "abfss://container@foo.bar/"
    error_message = "Expected the external location URL to be 'abfss://container@foo.bar/'"
  }
}

Debug Output

Output of TF_LOG=TRACE terraform test: gist

Expected Behavior

I would expect the test nothing_mocked_failure_shows_generated_id to fail, and the other tests to pass.

Actual Behavior

In addition to nothing_mocked_failure_shows_generated_id, external_location_url_mocked_unsuccessfully also fails:

  run "external_location_url_mocked_unsuccessfully"... fail
╷
│ Error: Invalid index
│ 
│   on variable_validation.tftest.hcl line 43, in run "external_location_url_mocked_unsuccessfully":
│   43:     condition = data.databricks_external_location.test.external_location_info[0].url == "abfss://container@foo.bar/"
│     ├────────────────
│     │ data.databricks_external_location.test.external_location_info is empty list of object
│ 
│ The given key does not identify an element in this collection value: the collection has no elements.
╵

Inspecting the data source value using an assertion which fails and printing the contents shows that external_location_url is not set:

╷
│ Error: Test assertion failed
│ 
│   on variable_validation.tftest.hcl line 43, in run "external_location_url_mocked_unsuccessfully":
│   43:     condition = length(data.databricks_external_location.test.external_location_info) > 0
│     ├────────────────
│ 
│ 
│ jsonencoded value of data:
│ {"external_location_info":[],"id":"c2rwnoaz","name":"test","provider_config":[]}

Steps to Reproduce

1. terraform test

Additional Context

My strong suspicion is that the type conversion of tuple to list is not occurring properly in the override_data block, and instead of raising an error, the default value is used (an empty collection, as defined in the docs).

An argument against this suspicion is that types seem to be enforced when setting properties in override_data blocks. When I set the id attribute to ["with-id-mocked"] in the test id_mocked_successfully, it does raise an error:

  run "id_mocked_successfully"... fail
╷
│ Error: Failed to compute attribute
│ 
│   with data.databricks_external_location.test,
│   on main.tf line 12, in data "databricks_external_location" "test":
│   12: data "databricks_external_location" "test" {
│ 
│ Terraform could not compute a value for the target type string with the mocked data defined at variable_validation.tftest.hcl:17,3-16 with the attribute ".id": string required, but have tuple.

However, I'm not sure whether [] is considered valid input for an list attribute during type checking.

When I inspect the contents of a data source successfully fetched from a live environment using terraform console, I can see that the external_location_info is defined with tolist():

> data.databricks_external_location.schema_locations
{
  "<redacted>" = {
    "external_location_info" = tolist([
      {
        "browse_only" = false
        "comment" = "<redacted>"
        "created_at" = 1760513678249
        "created_by" = "<redacted>"
        "credential_id" = "<redacted>"
        "credential_name" = "<redacted>"
        "enable_file_events" = false
        "encryption_details" = tolist([])
        "fallback" = false
        "file_event_queue" = tolist([])
        "isolation_mode" = "ISOLATION_MODE_OPEN"
        "metastore_id" = "<redacted>"
        "name" = "<redacted>"
        "owner" = "<redacted>"
        "read_only" = false
        "updated_at" = 1760513678675
        "updated_by" = "<redacted>"
        "url" = "abfss://<redacted>@<redacted>.dfs.core.windows.net/"
      },
    ])
    "id" = "<redacted>"
    "name" = "<redacted>"
    "provider_config" = tolist(null) /* of object */
  }
}

Since functions in override_data blocks are not permitted, I don't know how I can verify whether my suspicion is correct.

References

No response

Generative AI / LLM assisted development?

No response

[jbardin]

Hi @sanderginn,

I think what you are seeing is just another manifestation of #37862. The test framework can't assign values to a block schema, so it is ignored. Aside from the problems around blocks in general within the test framework however, I think there should probably be some sort of feedback if an override is not valid in some way.

Thanks!

[sanderginn]

Hi @jbardin ,

Thanks for your reply. I did see that issue, but my assessment is that it is describing a different scenario. That issue discusses not being able to override a block parameter on the data source definition, whereas my issue relates to overriding an exported attribute. Moreover, the data source docs describe this attribute as a type "array of objects".

Perhaps my understanding of how override_data is wrong? I assumed that it simply fills the response object with the values defined in override_data, which would lead me to believe there are no block definitions involved (since the type of external_location_info is an array).

[jbardin]

There is no "array" type in the Terraform language, so it must be referring to a list (and tuples are not part of the schema definition). The schema for this also can't be an attribute, because structural attributes did not exist in the legacy SDK which the DataBricks provider is still using (correction, it seems to be at least mostly using the framework). It's not very easy to point out the relevant source in the provider repo because of the abstractions they use for generating the schema, but you can see in the schema output that external_location_info is a list block.

    "attributes": {
      "id": {
        "type": "string",
        "description_kind": "plain",
        "optional": true,
        "computed": true
      },
      "name": {
        "type": "string",
        "description_kind": "plain",
        "required": true
      }
    },
    "block_types": {
      "external_location_info": {
        "nesting_mode": "list",
        "block": {
          "attributes": {
            "browse_only": {
              "type": "bool",
              "description_kind": "plain",
              "optional": true
            },
            ...

So external_location_info is a block, which the testing framework ignores in this case since it is not present in the configuration.

I agree that override_data should probably just be filling in the response data. What I think is happening here is that the test framework assumed that only computed attributes can produce values, and blocks cannot be computed. That assumption however does not hold for the legacy provider SDK.

[sanderginn]

I appreciate that you took the time to clarify this! I've been using Terraform since v0.12 and honestly never knew about the distinction between structural attributes and blocks. This specific data source seems to be a strange implementation to begin with. An external location can only have one set of values for the attributes inside of an external_location_info block, so it sounds to me like it should have been represented as a single nested block.

I do have one followup question: the docs do mention overriding computed values in repeated blocks. Is conceptually different from what I am trying to achieve?

[sanderginn]

Ah.. I see that the schema of aws_dynamodb_table defines the arn attribute of a replica block as computed:

      "replica": {
        "nesting_mode": "set",
        "block": {
          "attributes": {
            "arn": {
              "type": "string",
              "description_kind": "plain",
              "computed": true
            },

That isn't the case for url in external_location_info, so I guess that's why it won't register the override?

[jbardin]

This isn't quite the same situation as in the linked documentation, as that is talking about blocks which are present in the configuration, but contain computed attributes. A block is a description of the configuration syntax to decode, and does not represent a computed value on its own, notice the block description (either in the schema, or the code) has no computed flag at all.

A single object is better represented by a nested single block, but that value still cannot be computed. The use of a list or set container to have only a single object is a remnant of the legacy SDK and pre-0.12 Terraform which did not have that block type, nor did it have structural attributes.