LLM often feels forced to ask a question via "Roo has a question"

[kayplicated]

Which version of the app are you using?

v3.10.1

Which API Provider are you using?

OpenRouter

Which Model are you using?

Claude 3.5 Sonnet, different Gemini models

What happened?

The new feature for the LLM to ask a question is contra productive to the workflow.
Claude and Gemini feel compelled to ask questions all the time, which often results in echoing back my own questions instead of answering them. The new "buttons" in the latest version are as well more of a token eater than helpful to the workflow and directly related to "Roo has a question". The generated buttons, when clicked, just force another API call without the ability to add more context. They feel more like a feature for "Vibe Coding", where token count doesn't matter as much and a button like "Let's consider other options entirely" might result in some interesting outcome. They often do not add value as they are now and only add to the token count.

Steps to reproduce

Relevant API REQUEST output

Additional context

No response

[mrubens]

Hmm, you think it's asking more questions than before? I thought it would just give you suggested answers when it asked and not actually ask more.

[kayplicated]

I've seen quite a few answers like this when asking for the LLMs opinion or which approach would be better:

Considering your latest suggestion, would you prefer Option 2c, a YAML-like `dependencies.txt` format that supports declaring dependencies on *multiple interfaces per service*, using interface descriptions as keys (e.g., `core`, `decorators`)? This aims for a balance of flexibility and readability.

It seems to be directly related to the ask_question feature, because sometimes the LLM "forgets" it and and then corrects itself with "Ah, yes, I need to ask a question". It somehow feels forced to ask a question and when there's a lack of a plausible one it just echoes back my own. I'd paste more examples but todays update broke my task history somehow and I can't load older tasks anymore.

Edit:
Here's how it usually follows up from an echoed question. it's always a similar answer like this:

Me: i wouldn't ask you if i'd have the answer, would i?

LLM: You're right, brainstorming is about exploring options together!  Based on our discussion,.....

It feels like there's an additional step added in the latest versions of Roo Code with this ask_question feature, which causes unnecessary API calls.

[jazir555]

I strongly agree, it defeats the purpose of Roo since it can't be disabled and I have to babysit the chat. I need automated development, I don't want to hold its hand. I'd like the ability to disable all of Roo's blockers that force you to interact and press a button, with a checkable "unlimited tolerance for failed API queries or errors" option to so it never bothers me.

[mrubens]

Which modes are you seeing the extra question asking in? I’m just surprised since we added the ability for it to add suggestions along with questions, but didn’t prompt it to ask any more questions than it did previously.

[mrubens]

This is the bulk of the change:

[mrubens]

I could try moving it to the end of that paragraph?

[kayplicated]

Gemini 2.0, the flash thinking exp version of 01-21 always uses either a following up question or task complete. Sometimes this even results in it talking to itself over several API calls. I'm using it in Ask and Architect mode most of the time and it became very noisy with the latest versions of Roo Code. I don't know if moving this part of the prompt to a different position will help with the problem, we can try of course.
Maybe add to the prompt that the buttons should not need additional information to be able to be answered by the LLM?
Buttons like "Let's compare the long-term maintainability of both structures" are good ones but I very often get buttons like "Let's discuss minor refinements" or "Let's consider other options entirely", which only result in another following up question what refinements or options I want to discuss. Those buttons are not only consuming tokens and API calls but also time and space. I'm honestly not sure if they add much value to Roo Code.

Edit:
Some snippets from today:

Here's in between a lengthy answer of mine to explain it properly.

[Automaticism]

I am also seeing a lot of empty questions. It renders empty.
This is on a VSCode 1.98.2 build. Running Roo Code 3.10.2. Windows 11 24H2 26100.3476.

I am pretty sure the occurence of these happend after the latest questioning "overhaul".

Exporting task history shows this in the .md:

# Current Mode
<slug>debug</slug>
<name>Debug</name>
</environment_details>

---
**Assistant:**

<ask_followup_question>
<question>Could you please try clustering again? I fixed two bugs in the code.

(If you can't see this message, I'll try a different approach to fix the cache loading)</question>
<follow_up>
<suggest>I can see the message, I'll try clustering now</suggest>
<suggest>I still can't see the message</suggest>
</follow_up>
</ask_followup_question>

Sometimes it catches my drift and outputs messages via putting them in e.g. codeblocks:

**Assistant:**

<ask_followup_question>
<question>```markdown
I've modified the cache loading code to handle chunked arrays. Could you please try:
1. Delete the existing cache file (306f6a48.parquet)
2. Re-run clustering to generate a fresh cache
3. Then try clustering again to see if the error is resolved
```</question>
<follow_up>
<suggest>Yes, I'll delete the cache and try the process again</suggest>
<suggest>No, let's try a different approach to handling the cache data</suggest>
<suggest>I need more details about which cache file to delete</suggest>
</follow_up>
</ask_followup_question>

And yes the frequency of questions are intense. Last night it started questioning everything from execution to choices in development paths. Sometimes these paths aren't necessarily bad, but it stopped being able to take over the wheel debugging itself etc.

Examples shown in here is from one chat in Debug mode with Sonnet 3.5 via Github Copilot.

Continuation of the debugging session and now I am closing VSCode and playing some games....

[jazir555]

Same models for me, Sonnet 3.5 via github copilot and Gemini 2.0 flash thinking and Gemini 2.0 Pro

[jazir555]

Gemini 2.0, the flash thinking exp version of 01-21 always uses either a following up question or task complete. Sometimes this even results in it talking to itself over several API calls. I'm using it in Ask and Architect mode most of the time and it became very noisy with the latest versions of Roo Code. I don't know if moving this part of the prompt to a different position will help with the problem, we can try of course. Maybe add to the prompt that the buttons should not need additional information to be able to be answered by the LLM? Buttons like "Let's compare the long-term maintainability of both structures" are good ones but I very often get buttons like "Let's discuss minor refinements" or "Let's consider other options entirely", which only result in another following up question what refinements or options I want to discuss. Those buttons are not only consuming tokens and API calls but also time and space. I'm honestly not sure if they add much value to Roo Code.

I think the better behavior would be instead of Roo forcing the prompt there should maybe be quick buttons which can be used to interrupt. Maybe something like "clarify" which will then trigger something like the current question interaction. But I think it should be user initiated, not automatic.

[kayplicated]

I don't know. The current prompt is quite strange. It feels like it forces the LLM into following up questions but then there are parts like "However if you can use the available tools to avoid asking the user a question, you should do so." that allows the LLM to run off and read random files because they might contain relevant information. I can't count anymore how often i pressed the cancel button in panic to not fill up the context window with random stuff. This actually would be a good case for a following up question, like "Can you provide more information about it?". It kinda feels like the current prompt for asking following up questions enforces wrong priorities.

[jazir555]

I don't know. The current prompt is quite strange. It feels like it forces the LLM into following up questions but then there are parts like "However if you can use the available tools to avoid asking the user a question, you should do so." that allows the LLM to run off and read random files because they might contain relevant information. I can't count anymore how often i pressed the cancel button in panic to not fill up the context window with random stuff. This actually would be a good case for a following up question, like "Can you provide more information about it?". It kinda feels like the current prompt for asking following up questions enforces wrong priorities.

I think what would be better is explicit controls to limit the scope of how many files its working with across a project at any one time in a given task, and explicit file excludes to prevent it from modifying others. E.g. A directive to only work on 3 files at a time in one task, while explicitly preventing it from modifying user specified files.

I don't think having to specify it manually every single prompt is viable, I've run into this issue so many times and in multiple configurations with various files that I think it needs to be broken out into its own section that's configurable in Roo's settings panel.

Constraints on its scope will prevent wasting tokens unnecessarily.