-
Notifications
You must be signed in to change notification settings - Fork 2.6k
feat: add option to disable follow-up questions from LLM #6941
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
- Add alwaysAllowFollowupQuestions to SystemPromptSettings interface - Pass setting through generateSystemPrompt and Task.ts - Conditionally exclude ask_followup_question tool when disabled - Update UI description to clarify behavior when disabled Fixes #6940
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I reviewed my own code and found it acceptable. The bar was on the floor.
| "followupQuestions": { | ||
| "label": "Question", | ||
| "description": "Automatically select the first suggested answer for follow-up questions after the configured timeout", | ||
| "description": "Allow the AI to ask follow-up questions. When disabled, the AI will not have access to the question-asking tool and must proceed without clarification", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing internationalization for other locales - all other locale files (fr, de, es, ca, hi, id, it, ja, ko, nl, pl, pt-BR, ru, tr, vi, zh-CN, zh-TW) still have the old description for the follow-up questions setting. Could we update those as well to avoid confusion for non-English users?
| maxConcurrentFileReads: number | ||
| todoListEnabled: boolean | ||
| useAgentRules: boolean | ||
| alwaysAllowFollowupQuestions: boolean |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider adding a comment about backward compatibility here. Since this adds a new required property to the interface, existing saved states might not have it. While the code defaults to true in multiple places, documenting this would help future maintainers understand the migration strategy.
|
|
||
| // Conditionally exclude ask_followup_question if disabled in settings | ||
| if (settings?.alwaysAllowFollowupQuestions === false) { | ||
| tools.delete("ask_followup_question") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This implementation looks good! The tool is properly excluded when the setting is disabled. Consider adding a specific test case to verify that ask_followup_question is actually removed from the tools list when alwaysAllowFollowupQuestions is false.
|
This implementation is incomplete |
Summary
This PR addresses issue #6940 by adding a setting that allows users to completely disable the LLM's ability to ask follow-up questions.
Problem
Some LLMs (particularly GPT OSS) overuse the question-asking tool, repeatedly asking for confirmation instead of proceeding with tasks. This creates a frustrating user experience where the AI gets stuck in loops asking unnecessary questions.
Solution
alwaysAllowFollowupQuestionsto the system prompt settingsask_followup_questiontool is completely removed from the LLM's available toolsChanges
alwaysAllowFollowupQuestions: booleantoSystemPromptSettingsinterfacegetToolDescriptionsForModeto conditionally exclude theask_followup_questiontoolgenerateSystemPrompt.tsandTask.tsto pass the setting through the systemTesting
Fixes #6940
Important
Adds a setting to disable LLM follow-up questions, removing the
ask_followup_questiontool when disabled, with updates to code, tests, and UI.alwaysAllowFollowupQuestionssetting to disable LLM follow-up questions.ask_followup_questiontool is removed from LLM's tools.alwaysAllowFollowupQuestionstoSystemPromptSettingsintypes.ts.getToolDescriptionsForModeintools/index.tsto excludeask_followup_questionwhen disabled.generateSystemPrompt.tsandTask.tsto handle the new setting.system-prompt.spec.tsandcustom-instructions.spec.tsto include the new setting.settings.jsonto describe the new setting behavior.This description was created by
for 7639054. You can customize this summary. It will automatically update as commits are pushed.