CLOUDP-309577: Update curl Parameters for Improved Reliability
#636
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
This PR updates the
curlcommand parameters used in our Postman calls, based on the observations from our CI/CD pipeline from the last failure.Background
During our evaluation of last failure based on tracing we added, we discovered that postman in fact was overwhelmed and took entire 2 minutes to get error. I could not reproduce this locally :(
This resulted in only two requests being made due to the previous
--retry-max-timelimit of 300 seconds.Changes
To enhance the robustness and reliability of our calls, we have decided to adjust the parameters as follows:
Retry Attempts: Changing the
--retryoption to retry up to 7 times to finetune between instant failures and each request taking max 2 minutes.Updated Maximum Retry Time: increased it to
--retry-max-time 1200to allow more retries within the 20-minute window (1200 seconds).These changes lead to a potential maximum execution time of 7 attempts * 2 minutes per attempt + 7 * 30 seconds delay, which totals less than 1200 seconds.
Note
IMHO change is only temporary to help us with stability of OpenAPI job, while we are waiting for Postman support.