Skip to content

Fix API Gateway v1 external name oscillation causing perpetual delete/recreate#1975

Open
aditmeno wants to merge 1 commit intocrossplane-contrib:mainfrom
canary-technologies-corp:fix-apigateway-v1-external-name-oscillation
Open

Fix API Gateway v1 external name oscillation causing perpetual delete/recreate#1975
aditmeno wants to merge 1 commit intocrossplane-contrib:mainfrom
canary-technologies-corp:fix-apigateway-v1-external-name-oscillation

Conversation

@aditmeno
Copy link

@aditmeno aditmeno commented Feb 26, 2026

Summary

Fixes #1974

7 API Gateway v1 resources using FormattedIdentifierFromProvider("/", ...) suffer from an external-name annotation oscillation bug. Each reconciliation cycle the crossplane.io/external-name annotation changes, triggering perpetual delete/recreate cycles and AWS 429 throttling.

Root Cause

FormattedIdentifierFromProvider inherits GetExternalNameFn = IDAsExternalName (reads tfstate["id"] as-is) but overrides GetIDFn to join parameters with /. The Terraform AWS provider stores internal IDs in a different format than the import format:

Resource GetIDFn produces (slash) tfstate["id"] stores (internal)
aws_api_gateway_method {restApiId}/{resourceId}/{httpMethod} agm-{restApiId}-{resourceId}-{httpMethod}
aws_api_gateway_integration {restApiId}/{resourceId}/{httpMethod} agi-{restApiId}-{resourceId}-{httpMethod}
aws_api_gateway_stage {restApiId}/{stageName} ags-{restApiId}-{stageName}
aws_api_gateway_method_settings {restApiId}/{stageName}/{methodPath} {restApiId}-{stageName}-{methodPath}
aws_api_gateway_gateway_response {restApiId}/{responseType} aggr-{restApiId}-{responseType}
aws_api_gateway_integration_response {restApiId}/{resourceId}/{httpMethod}/{statusCode} agir-{restApiId}-{resourceId}-{httpMethod}-{statusCode}
aws_api_gateway_method_response {restApiId}/{resourceId}/{httpMethod}/{statusCode} agmr-{restApiId}-{resourceId}-{httpMethod}-{statusCode}

Since GetExternalNameFn returns the internal format (e.g., agm-abc-def-GET) but the controller expects the slash format (e.g., abc/def/GET), the annotation oscillates every reconciliation.

Fix

New helper apiGatewayFormattedIdentifier(prefix, keys...) that bypasses tfstate["id"] entirely and reads individual tfstate fields directly:

  • GetExternalNameFn: Reads individual tfstate fields (e.g., tfstate["rest_api_id"], tfstate["resource_id"], tfstate["http_method"]) and joins them with / separators. This avoids the tfstate["id"] format mismatch entirely.
  • GetIDFn: Reads the same fields from parameters, joins them with - separators, and prepends the prefix (e.g., agm-). This produces the internal format Terraform expects.

This approach is simpler and more robust than parsing the internal ID format — it reads canonical field values directly from tfstate rather than reverse-engineering the prefix/separator encoding.

Round-trip: GetExternalNameFn(tfstate) → slash format → (annotation) → GetIDFn(params) → internal format → (Terraform) → tfstate fields unchanged → GetExternalNameFn → same slash format. No oscillation.

Changes

  • config/externalname.go: Added apiGatewayFormattedIdentifier() helper; replaced all 7 affected resource entries.

Verification

  • go build ./config/... — passes
  • go vet ./config/... — passes

Full audit

All other API Gateway v1 resources were audited and confirmed unaffected — they either use config.IdentifierFromProvider (where tfstate["id"] is a bare ID or already matches the import format) or FormattedIdentifierFromProvider where the internal ID happens to match the slash format. Details in #1974.

@aditmeno
Copy link
Author

Tested this on my EKS cluster and verified fix

@erhancagirici erhancagirici force-pushed the fix-apigateway-v1-external-name-oscillation branch from 0933c82 to b8b1ff8 Compare March 13, 2026 11:51
Copy link
Collaborator

@erhancagirici erhancagirici left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aditmeno thanks for the contribution and handling this 🙌 Looks good already, left a few small comments to consider.
Would you be interested in testing again with the proposed changes?

Note: I rebased your PR through Github UI, though it might have messed the commit signatures. Feel free to re-rebase. Sorry for the inconvenience

@aditmeno aditmeno force-pushed the fix-apigateway-v1-external-name-oscillation branch from b8b1ff8 to 30a1d89 Compare March 14, 2026 06:17
…/recreate

The 7 API Gateway v1 resources using FormattedIdentifierFromProvider("/", ...)
had a mismatch between GetExternalNameFn (which reads tfstate["id"]) and
GetIDFn (which joins parameters with "/"). The Terraform AWS provider stores
internal IDs in a different format (prefixed, dash-separated) than the import
format (slash-separated), causing the external-name annotation to change every
reconciliation and triggering perpetual delete/recreate cycles and AWS 429
throttling.

Fixes: crossplane-contrib#1974
Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>
@aditmeno
Copy link
Author

@aditmeno thanks for the contribution and handling this 🙌 Looks good already, left a few small comments to consider. Would you be interested in testing again with the proposed changes?

Note: I rebased your PR through Github UI, though it might have messed the commit signatures. Feel free to re-rebase. Sorry for the inconvenience

Tested, no regression, PR ready for review @erhancagirici

@aditmeno aditmeno requested a review from erhancagirici March 14, 2026 07:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

API Gateway v1 resources have external name oscillation due to ID format mismatch

2 participants