[PLT-890] add legacy variable to tfstate service and create backends by juliareynolds-nava · Pull Request #212 · CMSgov/cdap

juliareynolds-nava · 2025-04-11T17:19:49Z

🎫 Ticket

https://jira.cms.gov/browse/PLT-890

🛠 Changes

backend files defining s3 state buckets for greenfield apps

ℹ️ Context

Add backends to the TF in the platform repo (each backend is a 2-line file for each of the vpcs). We expect to have 12 new VPCs.

🧪 Validation

Buckets are created in S3 with tfstate files for each new backend definition file.

jscott-nava · 2025-04-11T17:28:59Z

AB2D SBX TF plan is failing, looks like aws_iam_account_alias doesn't exist in that account?

gfreeman-navapbc

LGTM, but these buckets seem to only exist in the non-prod account atm, just something to be aware of

gfreeman-navapbc · 2025-04-11T19:39:26Z

LGTM, but these buckets seem to only exist in the non-prod account atm, just something to be aware of

I see now that there's a resource we're creating for these before the data call, disregard

gsf

Generally looks good! There is no ab2d-mgmt, bcda-mgmt, or dpc-mgmt, so those backends and related tfstate resources should be removed. Instead, there should be a cdap-mgmt backend and tfstate in the prod account.

gsf · 2025-04-14T13:54:46Z

terraform/modules/bucket/main.tf

-  bucket = "${data.aws_caller_identity.current.account_id}-bucket-access-logs"
+  bucket = (var.legacy == true ? "${data.aws_caller_identity.current.account_id}-bucket-access-logs" :
+    data.aws_iam_account_alias.current.account_alias == "aws-cms-oeda-bcda-non-prod" ? "bucket-access-logs-20250409172631068600000001" :
+      "bucket-access-logs-tbd")


The "tbd" can be replaced with the suffix for the access logs bucket created in the prod account.

gsf

Let's also switch out the deprecated dynamodb_table for use_lockfile in these new backends (see https://developer.hashicorp.com/terraform/language/backend/s3#state-locking). Those dynamodb tables will also need to be removed, and the tfstate terraform will need to be updated to avoid creating tables in greenfield.

…use_lockfile = true in tfstate/terraform

…erraform

gsf

Is there a reason why the aws-params-env-action changes are included in this PR?

Also, we need to keep the legacy ab2d-mgmt and dpc-mgmt backends. And all legacy backends should keep the dynamodb_table.

gsf · 2025-04-14T20:38:45Z

.github/workflows/github-actions-runner-apply.yml

          role-to-assume: arn:aws:iam::${{ secrets.BCDA_ACCOUNT }}:role/delegatedadmin/developer/bcda-mgmt-github-actions
          aws-region: ${{ vars.AWS_REGION }}
-      - run: terraform init -backend-config=../../backends/bcda-mgmt.s3.tfbackend
+      - run: terraform init -backend-config=../../backends/bcda-test.s3.tfbackend


This should be kept at bcda-mgmt for now to support the legacy accounts.

gsf · 2025-04-14T20:42:53Z

terraform/.terraform-version

@@ -1 +1 @@
-1.5.5
+1.10.0


As long as we're updating, we should switch to the latest patch under 1.10, which would be 1.10.5.

gsf · 2025-04-14T20:44:25Z

terraform/backends/ab2d-dev-gf.s3.tfbackend

@@ -0,0 +1 @@
+bucket         = "ab2d-dev-tfstate"


Shouldn't this have a suffix on the bucket name? Also, please run terraform fmt over these files.

gsf · 2025-04-14T20:44:51Z

terraform/backends/ab2d-dev.s3.tfbackend

@@ -1,2 +1 @@
 bucket         = "ab2d-dev-tfstate"
-dynamodb_table = "ab2d-dev-tfstate"


The dynamodb_table attribute should be kept for legacy backends.

Update your editor configuration to end files with a newline.

gsf · 2025-04-14T20:46:32Z

terraform/backends/ab2d-prod-gf.s3.tfbackend

@@ -0,0 +1,3 @@
+bucket         = "ab2d-prod-tfstate-20250411202936776600000001"
+
+


Drop these extra newlines.

gsf · 2025-04-14T20:52:21Z

terraform/services/tfstate/terraform.tf

  backend "s3" {
-    key = "tfstate/terraform.tfstate"
+    key          = "tfstate/terraform.tfstate"
+    use_lockfile = true


The use_lockfile attribute should be in the backend files for greenfield.

terraform/services/tfstate/main.tf

gsf · 2025-04-15T13:55:36Z

terraform/services/github-actions-runner/versions.tf

    key = "github-actions/terraform.tfstate"
  }
-  required_version = "~> 1.5.5"
+  required_version = "~> 1.10.0"


Drop this required_version

Remove mgmt env in greenfield

revert workflow changes

This reverts commit bf62750.

…es." This reverts commit f839348

gsf

Looks good and should be ready to merge once tfstate resources are all created (and extras cleaned up).

# Conflicts: # terraform/backends/ab2d-sandbox-gf.s3.tfbackend # terraform/backends/bcda-sandbox-gf.s3.tfbackend # terraform/backends/dpc-sandbox-gf.s3.tfbackend

gsf · 2025-04-16T21:55:33Z

terraform/services/tfstate/terraform.tf


 terraform {
-  # Comment out backend block and init without -backend-config for initial creation of resources
+  # # Comment out backend block and init without -backend-config for initial creation of resources


Suggested change

# # Comment out backend block and init without -backend-config for initial creation of resources

# Comment out backend block and init without -backend-config for initial creation of resources

gsf · 2025-04-17T16:29:50Z

terraform/modules/bucket/main.tf

+  bucket = data.aws_s3_bucket.bucket_access_logs.bucket
+
+  target_bucket = data.aws_s3_bucket.bucket_access_logs.bucket


This sets the wrong bucket as the source bucket for logging. Patch incoming.

gsf · 2025-04-17T16:31:19Z

terraform/modules/bucket/main.tf

 module "bucket_key" {
  source      = "../key"
-  name        = "${var.name}-bucket"
+  name        = data.aws_s3_bucket.bucket_access_logs.bucket


The key is for the bucket created in this module, not the access logs bucket. Patch incoming.

Also, I think we may run into issues if we start renaming all of the kms keys in legacy.

gsf

I think we're good to merge

add legacy variable to tfstate service and create backends

6f4e743

juliareynolds-nava requested a review from a team as a code owner April 11, 2025 17:19

juliareynolds-nava requested review from christopher-maboh, gfreeman-navapbc and jscott-nava April 11, 2025 17:20

jscott-nava approved these changes Apr 11, 2025

View reviewed changes

gfreeman-navapbc approved these changes Apr 11, 2025

View reviewed changes

gsf reviewed Apr 14, 2025

View reviewed changes

juliareynolds-nava added 9 commits April 14, 2025 12:12

add greenfield buckets

fa9ed38

remove dynamo table config

3f6a9eb

remove dynamo table config

4786972

remove dynamo table config

787ec9b

remove dynamo table config

ce06b3d

add use_lockfile to backend s3

ac2f8d7

add use_lockfile to backend s3. Upgrade to terraform 1.10 to support …

2ebbe71

…use_lockfile = true in tfstate/terraform

Upgrade to terraform 1.10 to support use_lockfile = true in tfstate/t…

ade7f0c

…erraform

Remove mgmt env for services

6e6030e

gsf reviewed Apr 14, 2025

View reviewed changes

terraform/services/tfstate/main.tf Show resolved Hide resolved

juliareynolds-nava added 2 commits April 14, 2025 15:02

Remove mgmt env for services

354264b

Remove mgmt env for services

65ce333

juliareynolds-nava marked this pull request as draft April 14, 2025 21:17

gsf reviewed Apr 15, 2025

View reviewed changes

juliareynolds-nava added 3 commits April 15, 2025 08:04

Only delete dynamo table for greenfield

2656185

Remove mgmt env in greenfield

restore table for legacy

7a64dfd

revert workflow changes

restore table for legacy mgmt

40e8284

juliareynolds-nava and others added 16 commits April 16, 2025 10:25

update sbx to sandbox

bf62750

update sbx to sandbox

56cf0c7

Revert "update sbx to sandbox"

4e8ee0c

This reverts commit bf62750.

Drop changes that should not be in this PR

09f23c7

Fix newline at end of file

2dd016a

Merge branch 'main' into jreynolds_plt_890_bucket_changes

93db60c

Add sandbox env as an option for tfstate

87b3761

added "sandbox" to environment options for services and modules.

f839348

correct terraform format

933835b

correct terraform format

e0b88e2

add mgmt for legacy backends

729b8f5

tf fmt

c55e86d

creating sandbox backends in non-prod

5be22e1

revert

417ce68

Revert "added "sandbox" to environment options for services and modul…

efd2d82

…es." This reverts commit f839348

Format gf backends

7fb6293

gsf reviewed Apr 16, 2025

View reviewed changes

juliareynolds-nava added 2 commits April 16, 2025 14:46

created prod backends

11a3b4f

# Conflicts: # terraform/backends/ab2d-sandbox-gf.s3.tfbackend # terraform/backends/bcda-sandbox-gf.s3.tfbackend # terraform/backends/dpc-sandbox-gf.s3.tfbackend

created prod backends

f7b6c26

gsf reviewed Apr 16, 2025

View reviewed changes

juliareynolds-nava added 2 commits April 16, 2025 15:55

move dynamodb table

79ed1c6

update ab2d dev

641930e

gsf reviewed Apr 17, 2025

View reviewed changes

gsf added 3 commits April 17, 2025 12:38

Fix references in bucket module

afc5a72

Drop extra comment hash

87af94c

Fix ordering in variable error message

4118497

gsf approved these changes Apr 17, 2025

View reviewed changes

juliareynolds-nava merged commit af97746 into main Apr 17, 2025
76 checks passed

juliareynolds-nava deleted the jreynolds_plt_890_bucket_changes branch April 17, 2025 17:39

		@@ -1,2 +1 @@
		bucket = "ab2d-dev-tfstate"
		dynamodb_table = "ab2d-dev-tfstate"

		@@ -0,0 +1,3 @@
		bucket = "ab2d-prod-tfstate-20250411202936776600000001"

	# # Comment out backend block and init without -backend-config for initial creation of resources
	# Comment out backend block and init without -backend-config for initial creation of resources

		bucket = data.aws_s3_bucket.bucket_access_logs.bucket

		target_bucket = data.aws_s3_bucket.bucket_access_logs.bucket

		@@ -1 +1 @@
		1.5.5
		1.10.0

Conversation

juliareynolds-nava commented Apr 11, 2025

🎫 Ticket

🛠 Changes

ℹ️ Context

🧪 Validation

Uh oh!

jscott-nava commented Apr 11, 2025

Uh oh!

gfreeman-navapbc left a comment

Choose a reason for hiding this comment

Uh oh!

gfreeman-navapbc commented Apr 11, 2025

Uh oh!

gsf left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gsf left a comment

Choose a reason for hiding this comment

Uh oh!

gsf left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gsf left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gsf left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants