Skip to content

Frequent tool use issues with Gemini flash 2.5 preview - 5.20 #3980

@Nolson37

Description

@Nolson37

App Version

3.18.3

API Provider

Google Gemini

Model Used

gemini-2.5-flash-preview-05-20:thinking

🔁 Steps to Reproduce

While use Orchestrator

  • instruct roo to implement a somewhat complex change (E.G. Research and implement open telemetry for the registration flow)
  • Interact with it as needed to get an architecture plan in place
  • Watch it as it tries to code

💥 Outcome Summary

Whats wrong:

  • Frequently fails to edit files, about as often as it succeeds. Often the issue is due to it not actually changing anything in the write operation despite multiple attempts
  • Frequent issues formatting json when using sequential thinking MCP (this is the ONLY model I've used yet that has this issue)
  • Frequently gets caught endlessly looping over the same read and then failure to write
  • Nearly unresponsive to interventional prompts
  • Absolutely incapable of effectively navigating the application with Playwright MCP (Again, first model I've used that struggles even close to this badly)

What's expected:
Given the benchmarking I've seen, I would expect this model to not be as good as Gemini 2.5 pro, or claud 3.7, etc, but thus far it's essentially incompetent. I believe it's likely that something in the integration of this new model could be the cause of the issue.

📄 Relevant Logs or Errors (Optional)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Issue/PR - TriageNew issue. Needs quick review to confirm validity and assign labels.bugSomething isn't working

    Type

    No type

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions