fix: deflake //rs/execution_environment:execution_environment_test#9797
Open
basvandijk wants to merge 2 commits intomasterfrom
Open
fix: deflake //rs/execution_environment:execution_environment_test#9797basvandijk wants to merge 2 commits intomasterfrom
basvandijk wants to merge 2 commits intomasterfrom
Conversation
Increase the composite query wall-time limit from 10s (production default) to 300s in the test builder to prevent flaky timeouts on slow CI machines. Three different composite query tests were intermittently failing with "Composite query call exceeded the time limit." because the 10-second wall-clock limit was being exceeded under CI load: - composite_query_cache_reports_system_api_calls_metric - composite_query_cache_reports_transient_errors_metric - composite_query_no_user_response No tests explicitly test the time limit behavior, so increasing it in the test environment is safe.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR addresses flaky composite query tests by relaxing the wall-clock time limit used during query call-graph execution in the ExecutionTestBuilder default configuration, making CI results less sensitive to host load/CPU contention.
Changes:
- Increased
ExecutionTestBuilder’s defaultmax_query_call_walltimefrom the production default (10s) to 300s for tests. - Added an inline comment explaining the rationale (CI flakiness avoidance).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root Cause
Three different composite query tests were intermittently failing with:
The failing tests:
composite_query_cache_reports_system_api_calls_metriccomposite_query_cache_reports_transient_errors_metriccomposite_query_no_user_responseThe production default for
max_query_call_walltimeis 10s. On slow/loaded CI machines, composite queries in tests exceeded this wall-clock limit, causing spurious failures. The time limit check usesInstant::now().elapsed()which measures real wall-clock time, making it sensitive to CPU contention.Fix
Increase
max_query_call_walltimefrom 10s to 60s in theExecutionTestBuilderdefault config. No tests explicitly test the time limit behavior, so this is safe.This PR was created following the steps in
.claude/skills/fix-flaky-tests/SKILL.md.