Conversation
3e34159 to
13d4519
Compare
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR centralizes timeout values and exposes an evaluation timeout flag for CLI runs:
- Adds an optional
timeoutparameter torun_evaluationand handlesTimeoutExpired. - Introduces
weco/constants.pywithDEFAULT_API_TIMEOUTand replaces hardcoded API timeouts. - Adds a
--eval-timeoutCLI argument and propagates it throughexecute_optimization.
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| weco/utils.py | Extended run_evaluation with a timeout parameter and exception handling. |
| weco/constants.py | New file defining DEFAULT_API_TIMEOUT. |
| weco/optimizer.py | Propagates eval_timeout and swaps hardcoded timeouts for DEFAULT_API_TIMEOUT. |
| weco/cli.py | Added --eval-timeout argument and pass-through to optimization function. |
| weco/api.py | Replaced hardcoded default timeouts with DEFAULT_API_TIMEOUT. |
| README.md | Documented the new --eval-timeout flag. |
Comments suppressed due to low confidence (2)
weco/utils.py:127
- The docstring for
run_evaluationshould be updated to describe the newtimeoutparameter and its behavior.
def run_evaluation(eval_command: str, timeout: int | None = None) -> str:
weco/utils.py:132
- New timeout logic in
run_evaluationshould have corresponding unit tests to verify both normal execution and timeout behavior.
result = subprocess.run(eval_command, shell=True, capture_output=True, text=True, check=False, timeout=timeout)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
DhruvSrikanth
approved these changes
Jul 8, 2025
Member
DhruvSrikanth
left a comment
There was a problem hiding this comment.
Looks good to me!
Would this require any changes to the onboarding copilot? Probably not since we would use the default value for eval-timeout which allows a unlimited time for the eval script's execution.
If this is the case, then feel free to merge @dexhunter
Member
Author
|
yes, timeout=None by default (unlimited) |
38ca262 to
b79b379
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
constants.pyto put values in a central place