Skip to content

feat: support multiple SSM parameters for large runner matcher configs (#4790) #4792

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open

feat: support multiple SSM parameters for large runner matcher configs (#4790) #4792

Changes from 2 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
2d5991f
feat(webhook): support multiple SSM parameters for runner matcher config
edersonbrilhante Sep 25, 2025
85df592
style: fix yarn format issues
edersonbrilhante Sep 26, 2025
4af61cf
fix: fix logic in loadMatcherConfig to pass all tests
edersonbrilhante Sep 26, 2025
9c22d0b
style: fix yarn format
edersonbrilhante Sep 26, 2025
214ca91
Merge branch 'main' into feat-ssm-runner-matcher-chunking
npalm Oct 3, 2025
06c1dcc
docs: auto update terraform docs
github-actions[bot] Oct 3, 2025
2259aaf
Update lambdas/functions/webhook/src/ConfigLoader.test.ts
npalm Oct 3, 2025
40e2a53
fix: fix broken logic to create multiple runner_matcher_config
edersonbrilhante Oct 3, 2025
bc9eb93
Merge branch 'main' into feat-ssm-runner-matcher-chunking
npalm Oct 4, 2025
b6e68d7
Merge branch 'main' into feat-ssm-runner-matcher-chunking
npalm Oct 4, 2025
367d51e
fix: handle runner config chunking for fresh SQS deployments
edersonbrilhante Oct 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions modules/webhook/webhook.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,11 @@ locals {
}

resource "aws_ssm_parameter" "runner_matcher_config" {
for_each = { for idx, val in local.matcher_chunks : idx => val }
count = length(local.matcher_chunks)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This count still results in my case in the following error with a large configuration:

Plan: 118 to add, 7 to change, 0 to destroy.
╷
│ Error: Invalid count argument
│
│   on ../../modules/webhook/webhook.tf line 27, in resource "aws_ssm_parameter" "runner_matcher_config":
│   27:   count = length(local.matcher_chunks)
│
│ The "count" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. To work around
│ this, use the -target argument to first apply only the resources that the count depends on.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@edersonbrilhante just tested with the example multi-runner in the repo. With about 4 extra config files with quite some labels. But ending up with the error above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@npalm I see the issue now — I was testing in an environment where the SQS queues were already created, so that was a blind spot. Testing with your example helped me pinpoint the problem.

The root cause is that the config relies on the SQS ARN. Since Terraform only knows the ARN at apply time, trying to calculate the string ahead of time for data evaluation doesn’t work.

My workaround is to precompute the worst-case scenario (max AWS partition, region, and queue name lengths) to estimate the maximum string length. From that, I calculate the number of chunks needed, create the resources using that chunk count, and then populate each resource with the real config split across the chunks at apply time.


name = "${var.ssm_paths.root}/${var.ssm_paths.webhook}/runner-matcher-config${length(local.matcher_chunks) > 1 ? "-${each.key}" : ""}"
name = "${var.ssm_paths.root}/${var.ssm_paths.webhook}/runner-matcher-config${length(local.matcher_chunks) > 1 ? "-${count.index}" : ""}"
type = "String"
value = each.value
value = local.matcher_chunks[count.index]
tier = var.matcher_config_parameter_store_tier
}

Expand Down