[Bug] elasticstack_elasticsearch_security_api_key resource "produced inconsistent result after apply"

[bplies-ATX]

Describe the bug
Attempting to create a elasticstack_elasticsearch_security_api_key succeeds in provisioning the key but results in the following error message

elasticstack_elasticsearch_security_api_key.jenkins_otel_api_key, provider
"provider["registry.terraform.io/elastic/elasticstack"]" produced an
unexpected new value: .role_descriptors: was
cty.StringVal("{"jenkins-otel":{"cluster":["monitor","read_ilm"],"indices":[{"names":["logs-apm.app*"],"privileges":["read","view_index_metadata"]}]}}"),
but now
cty.StringVal("{"jenkins-otel":{"cluster":["monitor","read_ilm"],"indices":[{"names":["logs-apm.app*"],"privileges":["read","view_index_metadata"],"allow_restricted_indices":false}]}}").

This is a bug in the provider, which should be reported in the provider's own
issue tracker.

So that is why I created this issue.

To Reproduce
Steps to reproduce the behavior:

1. TF configuration used '...'

resource "elasticstack_elasticsearch_security_api_key" "jenkins_otel_api_key" {
  name              = "Jenkins OTel API Key"
  role_descriptors  = jsonencode({
    jenkins-otel = {
      cluster = ["monitor", "read_ilm"],
      indices = [
        {
          names      = ["logs-apm.app*"], # Needed to show logs in Jenkins UI
          privileges = ["read", "view_index_metadata"]
        }
      ]
    }
}

2. TF operations to execute to get the error '...' [e.g terraform plan,terraform apply, terraform destroy]
   terraform plan produced a valid plan
   terraform apply created the resource but returns an error.

3. See the error in the output '...'

Expected behavior
Easy apply without error. Or improved documentation and validation for any required config.

Debug output
Not included.

Screenshots
N/A

Versions (please complete the following information):

• Terraform Version 1.8.1
• Provider version 0.11.16 (latest as of right now)
• Elasticsearch Version 8.17.1

Additional context
N/A

[georgepalmaris]

I'm having the same issue! Doesn't this just make creating api_keys with this provider impossible at the moment. I also tried on versions 0.8.x, 0.9.x & 0.11.x with the same issue.

[spr0ut]

For those using elastic cloud here is a workaround - not particularly elegant but gets the job done:

resource "null_resource" "api_key" {
  provisioner "local-exec" {
    command     = <<EOT
      role_descriptors_json=$(cat "role_descriptors.json" | jq -c .)
      curl -X POST "https://api.elastic-cloud.com/api/v1/deployments/${ec_deployment.this.id}/elasticsearch/main-elasticsearch/proxy/_security/api_key" \
        -H "Authorization: ApiKey $EC_API_KEY" \
        -H "X-Management-Request: true" \
        -H "Content-Type: application/json" \
        -d '{
              "name": "apikey",
              "expiration": "90d",
              "role_descriptors": '"$role_descriptors_json"'
            }' > "apikey.json"
    EOT
    interpreter = ["/bin/bash", "-c"]
  }
}

[bplies-ATX]

I had neglected to follow-up with a workaround. Just explicitly set allow_restricted_indices = false which should be optional and is the default anyway and that will make the provider happy wrt the "result after apply"

resource "elasticstack_elasticsearch_security_api_key" "jenkins_otel_api_key" {
  name              = "Jenkins OTel API Key"
  role_descriptors  = jsonencode({
    jenkins-otel = {
      cluster = ["monitor", "read_ilm"],
      indices = [
        {
          allow_restricted_indices = false, # << Just add this
          names      = ["logs-apm.app*"], # Needed to show logs in Jenkins UI
          privileges = ["read", "view_index_metadata"]
        }
      ]
    }
  })

  # No expiration
}

Ironically it turns out for the OpenTelemetry Jenkins plugin it cannot use an API key anyway and needed a basic username/password credential. So this example resource wasn't ultimately kept after all.

[tobio]

Duplicates #1236

[tobio]

This will be fixed in the next version via #1315