-
Notifications
You must be signed in to change notification settings - Fork 2.6k
feat: extend the tool to call the duplicate detection function to support pattern duplicate detection #5503
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
daniel-lxs
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey @aheizi Thank you for your contribution, I think this is an interesting approach to handling more complex forms of looping.
I left some suggestions that are worth taking a look at.
Let me know if you have any questions!
@daniel-lxs Thank you for the review. I have revised and submitted it. |
95de640 to
646bdd2
Compare
daniel-lxs
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @aheizi, I left a couple of suggestions to help improve your implementation.
Let me know what you think!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I noticed that after pattern detection triggers and resets, the next tool call still fails (line 468-469). Is this intentional?
If the goal of resetState() is to allow recovery after hitting a limit, shouldn't the next call be allowed? The current behavior could trap users in a situation where they can't recover from pattern detection.
Could you clarify the intended behavior here? If this is intentional, it might be worth adding a comment explaining why.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a comment explaining why.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider adding parameter validation in the constructor. This would prevent issue if someone passes negative values or extremely large values for historyLength or patternLimit.
For example:
constructor(consecutiveLimit: number = 3, historyLength: number = 20, patternLimit: number = 2) {
this.consecutiveIdenticalToolCallLimit = Math.max(0, consecutiveLimit)
this.historyMaxLength = Math.max(1, Math.min(historyLength, 1000)) // reasonable upper bound
this.patternRepetitionLimit = Math.max(0, patternLimit)
}This would prevent potential issues with invalid parameters.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When consecutiveIdenticalToolCallLimit is 0 (unlimited), the code skips all repetition checks including pattern detection. Is this intentional?
It seems like pattern repetition (ABABAB) could still be relevant even when consecutive repetitions (AAA) are unlimited. Should these two types of detection be independent?
If the current behavior is intentional, could you add a comment clarifying that setting consecutive limit to 0 disables ALL repetition detection, not just consecutive?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a comment clarifying that setting consecutive limit to 0 disables ALL repetition detection.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The magic number 5 for history retention should be extracted to a constant or made configurable for better maintainability.
Consider:
private static readonly HISTORY_RETENTION_ON_RESET = 5;
private resetState(): void {
this.consecutiveIdenticalToolCallCount = 0
this.previousToolCallJson = null
// Keep last N entries for context
this.toolCallHistory = this.toolCallHistory.slice(-ToolRepetitionDetector.HISTORY_RETENTION_ON_RESET)
}This makes it easier to adjust the value if needed and clarifies the intent.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok
646bdd2 to
372bf8c
Compare
…port pattern duplicate detection
|
@daniel-lxs Thank you for the review. I corrected some issues and adjusted some of the annotations to make them more accurate. |
372bf8c to
bf71632
Compare
|
@daniel-lxs There’s another issue: currently,
Since there’s no repetition at that point, the behavior feels inconsistent. There are three possible solutions:
I’m leaning towards option 2. If you’re also okay with it, I’ll add a user-facing message accordingly. Looking forward to your feedback. |
|
Hey @aheizi Good catch. I agree a message is the right approach here. That said, I think it should be clearer. “First-time invocation restriction is enabled” doesn’t make it obvious that setting consecutiveLimit to 1 means the tool will stop immediately after the first error, even if it’s not repeated. We should make that behavior explicit so users know it’s not about repetition at that point, but a strict 1-error threshold. |
Hey @daniel-lxs This is the performance of the todo list for the first time after it is set to 1: |
|
Or like this: |
|
Hey @aheizi, thank you for your work on this. After careful consideration, I believe this change could affect valid repetitive workflows that shouldn't be flagged as looping, such as editing files and running tests. This doesn't mean your implementation isn't solid or that we don't appreciate it. I'm going to close the PR for now, but I'm open to continuing the discussion. |
|
@daniel-lxs The problem of repeatedly calling tools that I am currently encountering is quite frequent. Although I am also using the cloude4 model, it is still very difficult to avoid the situation of pattern repetition. The situation you mentioned where there might be misjudgments when editing files and running tests, I think, also exists in the previous consecutive repetitive scenarios. Anyway, I still very much hope that this problem can be solved from an engineering perspective. |

Related GitHub Issue
Closes #5496
Description
The Tool Repetition Detector class has been enhanced and the pattern repetition detection function has been added, which can detect repetitive patterns like "ABCABC"
Test Procedure
unit test
Pre-Submission Checklist
Screenshots / Videos
Documentation Updates
Additional Notes
Get in Touch
aheizi
Important
Enhance
ToolRepetitionDetectorto detect pattern repetitions and add corresponding tests.ToolRepetitionDetectorto detect pattern repetitions like "ABCABC" inToolRepetitionDetector.ts.historyMaxLengthandpatternRepetitionLimitparameters in constructor.updateHistory(),detectPatternRepetition(), andhasRepeatingPattern()methods for pattern detection.resetState().ToolRepetitionDetector.spec.tsfor pattern detection (e.g., "ABABAB", "ABCABCABC").This description was created by
for bd331ed788f133f80b488cce4f914a2f47b8c99a. You can customize this summary. It will automatically update as commits are pushed.