feat: existing scc instance for DA

[jor2]

Description

Enable skipping SCC provisioning
terraform-ibm-modules/stack-retrieval-augmented-generation#131

Release required?

• No release
• Patch release (x.x.X)
• Minor release (x.X.x)
• Major release (X.x.x)

Release notes content

enable skipping SCC provisioning by providing option for passing existing scc instance id.

Run the pipeline

If the CI pipeline doesn't run when you create the PR, the PR requires a user with GitHub collaborators access to run the pipeline.

Run the CI pipeline when the PR is ready for review and you expect tests to pass. Add a comment to the PR with the following text:

/run pipeline

Checklist for reviewers

• If relevant, a test for the change is included or updated with this PR.
• If relevant, documentation for the change is included or updated with this PR.

For mergers

• Use a conventional commit message to set the release level. Follow the guidelines.
• Include information that users need to know about the PR in the commit message. The commit message becomes part of the GitHub release notes.
• Use the Squash and merge option.

[jor2]

Do we need to add an existing scc instance to permanent resources and specify it here?

[ocofaigh]

@jor2 no we don't need a permanent SCC instance. One can be provisioned in https://github.com/terraform-ibm-modules/stack-ibm-core-security-services/tree/main/tests/resources as part of the test if needed

[jor2]

/run pipeline

[jor2]

/run pipeline

[jor2]

/run pipeline

[jor2]

/run pipeline

[jor2]

/run pipeline

[jor2]

/run pipeline

[jor2]

/run pipeline

[jor2]

failure related to this again it seems, retrying

[jor2]

/run pipeline

[ocofaigh]

@jor2 hoping to do a release this week for core security services stack DA. Do we keep hitting the same issue in the tests here?

[jor2]

Getting this failure:

 2024/10/22 21:56:03 Terraform apply | Error: ReplaceIntegrationWithContext failed Provided endpoint value is not valid.

But I can see the endpoint is correct:

2024/10/22 21:53:42 Terraform plan |   # module.event_notifications[0].ibm_en_integration.en_kms_integration[0] will be created
 2024/10/22 21:53:42 Terraform plan |   + resource "ibm_en_integration" "en_kms_integration" {
 2024/10/22 21:53:42 Terraform plan |       + id             = (known after apply)
 2024/10/22 21:53:42 Terraform plan |       + instance_guid  = (known after apply)
 2024/10/22 21:53:42 Terraform plan |       + integration_id = (known after apply)
 2024/10/22 21:53:42 Terraform plan |       + type           = "hs-crypto"
 2024/10/22 21:53:42 Terraform plan |       + updated_at     = (known after apply)
 2024/10/22 21:53:42 Terraform plan | 
 2024/10/22 21:53:42 Terraform plan |       + metadata {
 2024/10/22 21:53:42 Terraform plan |           + crn         = "crn:v1:bluemix:public:hs-crypto:us-south:a/abac0df06b644a9cabc6e44f55b3880e:e6dce284-e80f-46e1-a3c1-830f7adff7a9::"
 2024/10/22 21:53:42 Terraform plan |           + endpoint    = "https://e6dce284-e80f-46e1-a3c1-830f7adff7a9.api.private.us-south.hs-crypto.appdomain.cloud/"
 2024/10/22 21:53:42 Terraform plan |           + root_key_id = (known after apply)
 2024/10/22 21:53:42 Terraform plan |         }
 2024/10/22 21:53:42 Terraform plan |     }

This matches the new endpoint schema introduced recently to hyper protect.

https://e6dce284-e80f-46e1-a3c1-830f7adff7a9.api.private.us-south.hs-crypto.appdomain.cloud

[jor2]

/run pipeline

[jor2]

/run pipeline

[jor2]

/run pipeline

[ocofaigh]

 2024/12/12 13:55:25 Terraform init | Error: Failed to install provider
 2024/12/12 13:55:25 Terraform init | 
 2024/12/12 13:55:25 Terraform init | Error while installing ibm-cloud/ibm v1.71.2: could not query provider
 2024/12/12 13:55:25 Terraform init | registry for registry.terraform.io/ibm-cloud/ibm: failed to retrieve
 2024/12/12 13:55:25 Terraform init | cryptographic signature for provider: the request failed after 2 attempts,
 2024/12/12 13:55:25 Terraform init | please try again later: Get
 2024/12/12 13:55:25 Terraform init | "https://github.com/IBM-Cloud/terraform-provider-ibm/releases/download/v1.71.2/terraform-provider-ibm_1.71.2_SHA256SUMS.sig":
 2024/12/12 13:55:25 Terraform init | net/http: request canceled (Client.Timeout exceeded while awaiting headers)

Retrying..

[ocofaigh]

/run pipeline

[jor2]

looks like a bug:

 2024/12/12 21:39:13 Terraform plan | Error: Invalid index
 2024/12/12 21:39:13 Terraform plan | 
 2024/12/12 21:39:13 Terraform plan |   on main.tf line 42, in module "existing_kms_key_crn_parser":
 2024/12/12 21:39:13 Terraform plan |   42:   crn     = var.existing_scc_cos_kms_key_crn != null ? var.existing_scc_cos_kms_key_crn : module.kms[0].keys[format("%!s(MISSING).%!s(MISSING)", local.scc_cos_key_ring_name, local.scc_cos_key_name)].crn
 2024/12/12 21:39:13 Terraform plan |     ├────────────────
 2024/12/12 21:39:13 Terraform plan |     │ module.kms is empty tuple

[jor2]

kms module has this count condition:

  count                       = var.existing_scc_cos_kms_key_crn != null || var.existing_scc_cos_bucket_name != null || var.existing_scc_instance_crn != null ? 0 : 1 # no need to create any KMS resources if passing an existing key or bucket, or SCC instance

We pass a value for existing_scc_cos_bucket_name so count is 0. the crn condition appears to be wrong and needs a rethink of the logic:

module "existing_kms_key_crn_parser" {
  count   = var.existing_scc_cos_kms_key_crn != null || var.existing_kms_instance_crn != null ? 1 : 0
  source  = "terraform-ibm-modules/common-utilities/ibm//modules/crn-parser"
  version = "1.1.0"
  crn     = var.existing_scc_cos_kms_key_crn != null ? var.existing_scc_cos_kms_key_crn : module.kms[0].keys[format("%s.%s", local.scc_cos_key_ring_name, local.scc_cos_key_name)].crn
}

source code is here

[jor2]

/run pipeline

[jor2]

/run pipeline

[jor2]

/run pipeline

[jor2]

will rerun after terraform-ibm-modules/terraform-ibm-scc-da#242 is merged

[jor2]

/run pipeline

[ocofaigh]

I don't think this will pass until provider releases SCC fix since event notification integration is enabled by default in this stack for SCC

[jor2]

/run pipeline

[jor2]

/run pipeline

[jor2]

/run pipeline

[ocofaigh]

No plans to implement