Skip to content

Parallel pipeline race: commit-filtered config-table lookup can return zero account mappings #1029

@karlzierhofer

Description

@karlzierhofer

Describe the bug
Overlapping AWSAccelerator-Pipeline executions can fail in later stages (for example identity-center) with Account Name not found for undefined.
The failure is caused by a race between commit-filtered account lookup and non-commit-isolated writes in the config DynamoDB table.

To Reproduce

  1. Use LZA with S3 config source and trigger deploy/execution A.
  2. While execution A is still running, trigger deploy/execution B with updated config artifact.
  3. Let execution A continue into later stages (e.g. identity-center).
  4. Observe failure in execution A with Account Name not found for undefined.

Verification done:

  • Failing build CONFIG_COMMIT_ID: OqydHA2sJxltNQqXANwuDsE.Rc5M8ncB
  • Querying config table for this commit returned:
    • mandatoryAccount: 0
    • workloadAccount: 0
  • Same account key existed in table with different commitId:
    • commitId: 1zt3ayXwCRwvwHJA9I3cK.I3YEvectIE

Expected behavior
Either:

  1. overlapping executions are explicitly prevented/unsupported at pipeline level, or
  2. config-table lookups are commit-isolated so each execution can always read a stable account mapping snapshot.

No execution should fail due to another in-progress execution updating commit metadata for shared keys.

Please complete the following information about the solution:

  • Version: 1.14.3
  • Region: eu-central-1 (home region), with global interactions in us-east-1
  • Was the solution modified from the version published on this repository? No (core LZA behavior observed)
  • If the answer to the previous question was yes, are the changes available on GitHub?
  • Have you checked your service quotas (https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html) for the services this solution uses? Yes
  • Were there any errors in the CloudWatch Logs? Yes (CodeBuild/ToolkitProject stage logs show account resolution failure)

Screenshots
N/A (logs available; can provide sanitized snippets on request).

Additional context
Relevant code paths:

  • throw site: source/packages/@aws-accelerator/config/lib/accounts-config.ts (getAccountId, getAccountNameById)
  • commit-filtered lookup: query-config-table.ts (contains(commitId, :commitId))
  • non-commit-isolated writes: load-config-table/index.ts (row key is dataType + acceleratorKey(email))

Related change introducing commit filter behavior:

  • e7f3bab (fix(lookup): filter out query with commitId)

This appears to be a concurrency race under overlapping pipeline runs, not a static config typo.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions