Skip to content

Commit ef3ec79

Browse files
PR review agent: make eval-risk approval policy repo-specific (#2254)
Co-authored-by: openhands <openhands@all-hands.dev>
1 parent fe9b8ff commit ef3ec79

File tree

2 files changed

+20
-15
lines changed

2 files changed

+20
-15
lines changed

.agents/skills/custom-codereview-guide.md

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,9 +13,27 @@ You are an expert code reviewer for the **OpenHands/software-agent-sdk** reposit
1313

1414
You have permission to **APPROVE** or **COMMENT** on PRs. Do not use REQUEST_CHANGES.
1515

16-
**Default to APPROVE**: If your review finds no issues at "important" level or higher, approve the PR. Minor suggestions or nitpicks alone are not sufficient reason to withhold approval.
16+
### Review decision policy (eval / benchmark risk)
1717

18-
**IMPORTANT: If you determine a PR is worth merging, you should approve it.** Don’t just say a PR is "worth merging" or "ready to merge" without actually submitting an approval. Your words and actions should be consistent.
18+
Do **NOT** submit an **APPROVE** review when the PR changes agent behavior or anything
19+
that could plausibly affect benchmark/evaluation performance.
20+
21+
Examples include: prompt templates, tool calling/execution, planning/loop logic,
22+
memory/condenser behavior, terminal/stdin/stdout handling, or evaluation harness code.
23+
24+
If a PR is in this category (or you are uncertain), leave a **COMMENT** review and
25+
explicitly flag it for a human maintainer to decide after running lightweight evals.
26+
27+
### Default approval policy
28+
29+
**Default to APPROVE**: If your review finds no issues at "important" level or higher,
30+
approve the PR. Minor suggestions or nitpicks alone are not sufficient reason to
31+
withhold approval.
32+
33+
**IMPORTANT:** If you determine a PR is worth merging **and it is not in the eval-risk
34+
category above**, you should approve it. Don’t just say a PR is "worth merging" or
35+
"ready to merge" without actually submitting an approval. Your words and actions should
36+
be consistent.
1937

2038
### When to APPROVE
2139

examples/03_github_workflows/02_pr_review/prompt.py

Lines changed: 0 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -39,19 +39,6 @@
3939
When posting a review, keep the review body brief unless your active review
4040
instructions require a longer structured format.
4141
42-
## Review decision policy (eval / benchmark risk)
43-
44-
You MAY approve clearly low-risk changes (docs, typo fixes, formatting, or
45-
pure refactors with no behavior changes).
46-
47-
Do NOT submit an APPROVE review when the PR changes agent behavior or anything
48-
that affects benchmark/evaluation performance.
49-
Examples include: prompt templates, tool calling/execution, planning/loop logic,
50-
memory/condenser behavior, terminal/stdin/stdout handling, or evaluation harness code.
51-
52-
If a PR is in this category (or you are uncertain), leave a COMMENTED review and
53-
explicitly flag it for a human maintainer to decide after running lightweight evals.
54-
5542
Review the PR changes below and identify issues that need to be addressed.
5643
5744
## Pull Request Information

0 commit comments

Comments
 (0)