[Bug] (azure_metrics integration) Provider produced inconsistent result after apply: .vars_json: inconsistent values for sensitive attribute

[x-dean]

Provider produces inconsistent result after apply: .vars_json: inconsistent values for sensitive attribute

Description

The Elasticstack Terraform provider produces an "inconsistent values for sensitive attribute" error when using vars_json with sensitive values in elasticstack_fleet_integration_policy resources. This issue specifically affects the azure_metrics integration but NOT the azure_openai integration, even when using identical sensitive value patterns.

Steps to Reproduce

1. Create an elasticstack_fleet_integration_policy with vars_json containing sensitive values
2. Run terraform plan - succeeds, shows vars_json = (sensitive value)
3. Run terraform apply - fails with inconsistent values error

Expected Behavior

Terraform should handle the conversion of sensitive values to secret references without state comparison issues.

Actual Behavior

Provider converts sensitive values to secret references during apply, but Terraform's state comparison fails because:

• Plan value: Single sensitive value (sensitive value)
• Apply value: Mixed object with strings + secret references

Configuration Examples

FAILS: azure_metrics with vars_json

# Complete example that fails
locals {
  test_client_id       = "test-client-id"
  test_tenant_id       = "test-tenant-id"
  test_client_secret   = "sensitive-secret-value"
  test_subscription_id = "test-subscription-id"
}

azure_metrics_test = {
    integration_name : "azure_metrics",
    version : "1.10.0"
    "vars_json" = {
    client_id       = local.test_client_id,
    tenant_id       = local.test_tenant_id,
    client_secret   = local.test_client_secret,
    subscription_id = local.test_subscription_id,
    }
    "inputs" = jsondecode(file("agent_policies/azure/azure_metrics_searchai.json"))
}


azure/azure_metrics_searchai.json

{
  "monitor-azure/metrics": {
    "enabled": true,
    "streams": {
      "azure.monitor": {
        "enabled": true,
        "vars": {
          "period": "300s",
          "resources": "- resource_query: \"resourceType eq 'Microsoft.Search/searchServices'\"\n  metrics:\n  - name: [\"DocumentsProcessedCount\", \"SearchLatency\", \"SearchQueriesPerSecond\", \"ThrottledSearchQueriesPercentage\"]\n    namespace: \"Microsoft.Search/searchServices\""
        }
      }
    }
  },
  "compute_vm-azure/metrics": {
    "enabled": false,
    "streams": {
      "azure.compute_vm": {
        "enabled": false,
        "vars": {
          "period": "300s",
          "resource_groups": [],
          "resource_ids": []
        }
      }
    }
  },
  "compute_vm_scaleset-azure/metrics": {
    "enabled": false,
    "streams": {
      "azure.compute_vm_scaleset": {
        "enabled": false,
        "vars": {
          "period": "300s",
          "resource_groups": [],
          "resource_ids": []
        }
      }
    }
  },
  "container_registry-azure/metrics": {
    "enabled": false,
    "streams": {
      "azure.container_registry": {
        "enabled": false,
        "vars": {
          "period": "300s",
          "resource_groups": [],
          "resource_ids": []
        }
      }
    }
  },
  "container_instance-azure/metrics": {
    "enabled": false,
    "streams": {
      "azure.container_instance": {
        "enabled": false,
        "vars": {
          "period": "300s",
          "resource_groups": [],
          "resource_ids": []
        }
      }
    }
  },
  "container_service-azure/metrics": {
    "enabled": false,
    "streams": {
      "azure.container_service": {
        "enabled": false,
        "vars": {
          "period": "300s",
          "resource_groups": [],
          "resource_ids": []
        }
      }
    }
  },
  "database_account-azure/metrics": {
    "enabled": false,
    "streams": {
      "azure.database_account": {
        "enabled": false,
        "vars": {
          "period": "300s",
          "resource_groups": [],
          "resource_ids": []
        }
      }
    }
  },
  "storage_account-azure/metrics": {
    "enabled": false,
    "streams": {
      "azure.storage_account": {
        "enabled": false,
        "vars": {
          "period": "300s",
          "resource_groups": [],
          "resource_ids": [],
          "service_types": []
        }
      }
    }
  }
}

WORKS: azure_openai with templatefile (same sensitive values)

# Complete example that works
locals {
  test_client_id       = "test-client-id"
  test_tenant_id       = "test-tenant-id"
  test_client_secret   = "sensitive-secret-value"  # Same sensitive value, works fine
  test_subscription_id = "test-subscription-id"
  eventhub_logs        = "eh-val-logs"
  eventhub_consumer_group = "obs-p"
  eventhub_logs_connection = "sensitive-connection-string"
  eventhub_storage_account_t = "testoffsets"
  storage_account_key_t = "sensitive-storage-key"
}

azure_openai_test = {
    integration_name : "azure_openai",
    version : "1.9.0"
    "inputs" = jsondecode(templatefile("agent_policies/azure/azure_openai.json", {
    client_id           = local.test_client_id,
    tenant_id           = local.test_tenant_id,
    client_secret       = local.test_client_secret,
    subscription_id     = local.test_subscription_id,
    eventhub            = local.eventhub_logs,
    consumer_group      = local.eventhub_consumer_group,
    connection_string   = local.eventhub_logs_connection,
    storage_account     = local.eventhub_storage_account_t,
    storage_account_key = local.storage_account_key_t
    }))
}

azure_openai.json

{
  "azure_open_ai-azure-eventhub": {
    "enabled": true,
    "streams": {
      "azure_openai.logs": {
        "enabled": true,
        "vars": {
          "eventhub": "${eventhub}",
          "consumer_group": "${consumer_group}",
          "connection_string": "${connection_string}",
          "storage_account": "${storage_account}",
          "storage_account_key": "${storage_account_key}",
          "preserve_original_event": false,
          "tags": [
            "azure-openai-logs-nonprod",
            "forwarded"
          ],
          "sanitize_newlines": true,
          "sanitize_singlequotes": true
        }
      }
    }
  },
  "azure_open_ai-azure/metrics": {
    "enabled": true,
    "streams": {
      "azure.open_ai": {
        "enabled": true,
        "vars": {
          "client_id": "${client_id}",
          "client_secret": "${client_secret}",
          "tenant_id": "${tenant_id}",
          "subscription_id": "${subscription_id}",
          "refresh_list_interval": "600s",
          "period": "300s",
          "resource_groups": [],
          "resource_ids": []
        }
      }
    }
  }
}

Error Message

Error: Provider produced inconsistent result after apply

When applying changes to elasticstack_fleet_integration_policy.test, provider
"registry.terraform.io/elastic/elasticstack" produced an unexpected new value: 
.vars_json: inconsistent values for sensitive attribute.

This is a bug in the provider, which should be reported in the provider's own issue tracker.

Plan Output

+ vars_json = (sensitive value)

Current State After Apply (DEBUG)

"vars": {
  "client_id": "test-client-id",
  "client_secret": {
    "id": "5QHPo5kBMigqx2DEelaT",
    "isSecretRef": true
  },
  "tenant_id": "test-tenant-id",
  "subscription_id": "test-subscription-id"
}

Environment

• Terraform v1.11.2
• provider registry.terraform.io/elastic/elasticstack v0.11.15
• Elasticsearch Version: 9.0.3

What We Tried

1. Template Variables in JSON (Failed)

# Tried embedding credentials in JSON template like azure_openai
"inputs" = jsondecode(templatefile("azure_metrics.json", {
  client_id = local.test_client_id,
  client_secret = local.test_client_secret,
  # ... other vars
}))

Result: Provider doesn't recognize template variables in azure_metrics JSON files. When we add ${client_id}, ${client_secret}, etc. to the azure_metrics JSON template, the provider fails with "Variable not found" errors, unlike azure_openai which works perfectly with template variables.

2. vars_json with Local Variables (Failed)

vars_json = {
  client_id = local.test_client_id,
  client_secret = local.test_client_secret,  # Same pattern as azure_openai
  tenant_id = local.test_tenant_id,
  subscription_id = local.test_subscription_id,
}

Result: Same "inconsistent values for sensitive attribute" error.

3. Direct Key Vault References (Failed)

vars_json = {
  client_secret = data.azurerm_key_vault_secret.all["secret-name"].value
}

Result: Same error - provider still converts to secret references inconsistently.

Key Differences

• azure_openai:
  • Uses streams_json with embedded credentials via templatefile() → Works fine
  • Supports template variables (${client_id}, ${client_secret}, etc.) in JSON → Works fine
• azure_metrics:
  • Uses vars_json with credentials → Fails with provider bug
  • Does NOT support template variables in JSON → Fails with "Variable not found" errors
  • Forced to use vars_json approach which has the sensitive value bug

Additional Context

• The provider selectively converts only some sensitive values to secret references
• Creates mixed-type objects that break Terraform's state comparison