Conversation
factory-nizar
force-pushed
the
feat/review-suggestion-blocks
branch
2 times, most recently
from
7f1cb65 to
987bf4d
Compare
|
Droid finished @factory-nizar's task —— View job Droid reviewed PR #52 — 1 finding approved (P1: blanket anchor-content restriction suppresses replacement suggestions), 1 rejected (speculative snippet-cap conflict). View review |
|
btw @jonathan-factory, @lukealvoeiro was suggesting 250 lines might be too big. I guess most suggestions even if they had egregious white space issues should not be more than sub 100? |
![factory-droid[bot]](https://avatars.githubusercontent.com/in/358017?s=60&v=4){kind=small}
There was a problem hiding this comment.
One approved finding: the blanket "do not change the anchor line content" instruction at line 352 contradicts the <replacement code> template and the suggestion blocks Rules section (which correctly scopes the unchanged-anchor restriction to insert-only suggestions). This would suppress the most common type of suggestion—single-line replacements. The suggested fix properly narrows the restriction to insert-only cases.
Yeah |
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> fix(review): escape suggestion blocks in prompt Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> fix(review): enforce suggestion-only replacements Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> update
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
factory-nizar
force-pushed
the
feat/review-suggestion-blocks
branch
from
916f6dc to
498436d
Compare
…ions parameter Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Benchmark: v3 baseline vs suggestion blocks + dedup (50 PRs, 5 repos)
+5 TPs, +23 FPs. Recall up 3.2pp, precision down 7.4pp, F1 down 2.1pp. |
| ### Deduplication (STRICT) | ||
|
|
||
| Before approving a candidate, check for duplicates: | ||
| 1. **Among candidates**: If two or more candidates describe the same underlying bug (same root cause, even if anchored to different lines or worded differently), approve only the ONE with the best anchor and clearest explanation. Reject the rest with reason "duplicate of candidate N". |
There was a problem hiding this comment.
Should we also include this dedupe block in the core review skill?
There was a problem hiding this comment.
yup! I can make sure it's there when it's moved to mono-repo ( if it's not there)
2 participants
Add review-bot instructions to include suggestion blocks when the fix is high-confidence, using RIGHT-side anchors and a 250-line cap.
There's a bit of repetition in the prompts but feel free to correct me if anything is not relevant:
candidate: generates comment bodies, so would naturally want to add suggestions there as well
Validator: enforcement, reject if anchor changes
Review prompt: needed in the case a single pass is enabled
examples
or in this pr we are in currently
I did focus on anchoring on the right, since applying suggestions on deleted lines is not currently supported
Followup: adding guidance about including suggestions or not based on a boolean in this pr is not super important if we are moving the prompts to skills, but might be a nice in between till I get everything working.
Closes FAC-16670