-
Notifications
You must be signed in to change notification settings - Fork 2.6k
Closed
Labels
Issue/PR - TriageNew issue. Needs quick review to confirm validity and assign labels.New issue. Needs quick review to confirm validity and assign labels.bugSomething isn't workingSomething isn't working
Description
App Version
3.18.3
API Provider
Google Gemini
Model Used
gemini-2.5-flash-preview-05-20:thinking
🔁 Steps to Reproduce
While use Orchestrator
- instruct roo to implement a somewhat complex change (E.G. Research and implement open telemetry for the registration flow)
- Interact with it as needed to get an architecture plan in place
- Watch it as it tries to code
💥 Outcome Summary
Whats wrong:
- Frequently fails to edit files, about as often as it succeeds. Often the issue is due to it not actually changing anything in the write operation despite multiple attempts
- Frequent issues formatting json when using sequential thinking MCP (this is the ONLY model I've used yet that has this issue)
- Frequently gets caught endlessly looping over the same read and then failure to write
- Nearly unresponsive to interventional prompts
- Absolutely incapable of effectively navigating the application with Playwright MCP (Again, first model I've used that struggles even close to this badly)
What's expected:
Given the benchmarking I've seen, I would expect this model to not be as good as Gemini 2.5 pro, or claud 3.7, etc, but thus far it's essentially incompetent. I believe it's likely that something in the integration of this new model could be the cause of the issue.
📄 Relevant Logs or Errors (Optional)
Metadata
Metadata
Assignees
Labels
Issue/PR - TriageNew issue. Needs quick review to confirm validity and assign labels.New issue. Needs quick review to confirm validity and assign labels.bugSomething isn't workingSomething isn't working
Type
Projects
Status
Done