Avoid using d.Get("task") for each task in appflow_flow #44543
+25
−10
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The original implementation called d.Get("task") for each task's source_fields attribute.
As a result, for each task, d.Get("task") is called - which turns out to be expensive.
For example for ~350 tasks, this seemed to take about 15 minutes.
Profiling for this original behavoir can be seen in the PR.
This change introduces a new single_task_flag attribute, which is computed once during the read operation and this is used to determine if a single task is present (which is what the original result of d.Get("task") was used for.
From local testing of the use-case, this completely fixes the slow-down
Community GuidelinesThis comment is added to every new Pull Request to provide quick reference to how the Terraform AWS Provider is maintained. Please review the information below, and thank you for contributing to the community that keeps the provider thriving! 🚀 Voting for Prioritization
Pull Request Authors
|
|
Flame graph from original: |
github-actions
bot
added
needs-triage
|
✅ Thank you for correcting the previously detected issues! The maintainers appreciate your efforts to make the review process as smooth as possible. |
github-actions
bot
added
the
tests
MatthewJohn
force-pushed
the
fix-appflow-flow-slow-task
branch
from
dbad771 to
a1efbd6
Compare
…w_flow is set correctly
MatthewJohn
force-pushed
the
fix-appflow-flow-slow-task
branch
from
a1efbd6 to
47da76d
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rollback Plan
A new attribute has been added, but believe this can just be reverted.
Changes to Security Controls
No
Description
The original implementation called d.Get("task") for each task's source_fields attribute. As a result, for each task, d.Get("task") is called - which turns out to be expensive. For example for ~350 tasks, this seemed to take about 15 minutes. Profiling for this original behavoir can be seen in the PR.
This change introduces a new single_task_flag attribute, which is computed once during the read operation and this is used to determine if a single task is present (which is what the original result of d.Get("task") was used for. From local testing of the use-case, this completely fixes the slow-down
Relations
Closes #0000
References
Output from Acceptance Testing
I don't have access to an AWS account, so I'm not fully sure how I can run these tests :/
I'm also not sure if I should be adding test cases?