In-Editor Browser Improvements #1601

afshawnlotfi · 2025-03-12T18:18:10Z

Summary

This PR refactors and improves the browser connection and management functionality in the Roo Code extension. It simplifies the browser discovery process, adds better tab management, and fixes bugs related to browser interactions.

Changes

New Features

Added a new hover browser action for more sophisticated UI interactions
Implemented smarter tab management that reuses tabs for the same domain
Added support for cached Chrome host URLs to improve connection reliability

Code Improvements

Refactored browser discovery to be more reliable and simpler
Consolidated browser connection logic into more modular, reusable functions
Improved tab management to reuse existing tabs when navigating to the same domain
Added caching of successful Chrome host URLs to improve reconnection reliability
Removed redundant code in the browser discovery process

Bug Fixes

Fixed potential "used before assigned" errors by properly initializing browserActionResult with an empty object
Added null checks with optional chaining for browser action results to prevent runtime errors
Improved error handling throughout the browser connection process

Testing

The changes have been tested with both local and remote browser connections across different environments. The new tab management functionality has been verified to work correctly with various navigation patterns.

Important

Enhances browser connection and management in Roo Code extension with new features, refactored logic, and improved error handling.

Behavior:
- Adds hover action to browser interactions in BrowserSession.ts.
- Implements smarter tab management in BrowserSession.ts to reuse tabs for the same domain.
- Supports cached Chrome host URLs for improved connection reliability in ClineProvider.ts and BrowserSession.ts.
Code Improvements:
- Refactors browser discovery and connection logic in ClineProvider.ts and browserDiscovery.ts.
- Consolidates browser connection logic into modular functions in BrowserSession.ts.
- Removes redundant code in browser discovery process in ClineProvider.ts.
Bug Fixes:
- Fixes "used before assigned" errors by initializing browserActionResult in Cline.ts.
- Adds null checks with optional chaining for browser action results in Cline.ts.
- Improves error handling in browser connection process in ClineProvider.ts and BrowserSession.ts.
Testing:
- Updates tests in ClineProvider.test.ts to cover new browser connection features and error handling.

^{This description was created by}^{for 5bec372. It will automatically update as commits are pushed.}

* Added multi-tab remote Chrome support * Added support for hover * Properly caching remote browser host in global state * Cleanup functions

changeset-bot · 2025-03-12T18:18:13Z

⚠️ No Changeset found

Latest commit: 8810d67

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

ellipsis-dev · 2025-03-12T18:19:23Z

After reviewing the changes in this pull request, it appears that the modifications are focused on improving browser session management and Chrome host discovery. The changes across the files seem to be interconnected and related to a common theme of browser improvements. Therefore, splitting the pull request into smaller ones may not be necessary as the changes are logically grouped together. If you believe there are distinct groups of changes that could be separated for better reviewability, please consider splitting them into separate pull requests.

ellipsis-dev · 2025-03-12T18:21:55Z

src/services/browser/BrowserSession.ts

-			return
+
+			// Cache the successful endpoint
+			console.log(`Connected to remote browser at ${chromeHostUrl}`)


Consider using a structured logging mechanism instead of plain console.log. This enables better log management and debugging in production.

ellipsis-dev · 2025-03-12T18:21:55Z

src/services/browser/browserDiscovery.ts

+		const response = await axios.get(`${chromeHostUrl}/json/version`, { timeout: 1000 })
 		const data = response.data
-		return { endpoint: data.webSocketDebuggerUrl, ip: ipAddress }
+		return true


The tryChromeHostUrl function returns true without validating response data. Consider checking if the expected property (e.g., the WebSocket URL) exists for more robust connectivity checks.

Suggested change

return true

return 'webSocketDebuggerUrl' in data

…wser-improvements

src/core/webview/ClineProvider.ts

* Parallel tasks * Handle null [x]ModelInfo types, recover from settings schema parse errors (#2064) * Require exactly node v20.18.1 (#2065) * Get rid of runner and all CJS stuff * Add an activation command for other extensions (#2073) Since VS Code provides no actionEvents that trigger when a specific extension activates, extensions can't activate only once Roo is activated and ready. This commit introduces a new roo-cline.activationCompleted command and fires it at the very end of extension activation. Other extensions that use Roo's exported APIs can now activate when they see this event. Co-authored-by: Greg Taylor <[email protected]> * Update bug_report.yml (#1994) * Update bug_report.yml * Update bug_report.yml --------- Co-authored-by: Matt Rubens <[email protected]> * Fix mention file name is not fully displayed (#2026) Fixed path leading character handling to preserve language characters and remove specific punctuation marks * In-Editor Browser Improvements (#1601) * Browser Automation Improvements * Added multi-tab remote Chrome support * Added support for hover * Properly caching remote browser host in global state * Cleanup functions * Updated for changes after merge * Added www. exception for common tabs * Update src/core/webview/ClineProvider.ts * Revert README changes --------- Co-authored-by: Matt Rubens <[email protected]> * Add missing awaits on refactored tools (#2078) * More progress * Added Gemini 2.5 Pro model to GCP Vertex AI Provider (#2079) * More progress * Display info about partial reads in chat row (#2080) * More progress * Update contributors list (#2023) docs: update contributors list [skip ci] Co-authored-by: mrubens <[email protected]> * More progress * Handle multiple API pages worth of contributors (#2088) * Update contributors list (#2089) docs: update contributors list [skip ci] Co-authored-by: mrubens <[email protected]> * Refactor/cline.ts/list files (#2067) * "Refactor list_files tool to separate module (#2057)" * Await --------- Co-authored-by: Matt Rubens <[email protected]> * Remove switch from tools (#2091) * HandleError returns a promise * Remove unnecessary switch from tools * Fix a type in the listFiles tool (#2092) * Move write_to_file to a tool file (#2093) * Move apply_diff to a tool file (#2094) * Move insert_content to a tool file (#2095) * Move search_and_replace to a tool file (#2096) * Move list_code_definition_names to a tool file (#2097) * Move search_files to a tool file (#2098) * Move browser_action to a tool file (#2099) * Move execute_command to a tool file (#2100) * Move use_mcp_tool and access_mcp_resource to tool files (#2101) * Move new_task, switch_mode, attempt_completion, and ask_followup_question to tool files (#2102) * Do a little code cleanup now that tools are refactored out (#2103) * Clean up the way we compute the current diff strategy (#2049) * Clean up the way we compute the current diff strategy * Add changeset * i18n Add translations for task pinning and unpinning (#2109) Signed-off-by: feifei <[email protected]> * Remove the ask promise error (#2107) * Remove the ask promise error * Send start_time along with checkpoints and use it to position in the messages * Link to the settings page from the auto approve toolbar (#2111) * Link to provider docs from the API options (#2112) * Revert "Remove the ask promise error" (#2117) Revert "Remove the ask promise error (#2107)" This reverts commit 712ca71. * Parse the retry out of the gemini 429 response (#2118) * Fix Gemini command escaping (#2120) * Fix switching profiles to ensure only the selected profile is switche… (#2119) Fix switching profiles to ensure only the selected profile is switched for all modes, and optimize certain i18n keys. Signed-off-by: feifei <[email protected]> * Tweaks to support prompt line number format (#2121) * Remove the ask promise error (#2123) * Stronger typing * Added Gemini 2.5 Pro model to GCP Vertex AI Provider (#2079) * Display info about partial reads in chat row (#2080) * Update contributors list (#2023) docs: update contributors list [skip ci] Co-authored-by: mrubens <[email protected]> * Handle multiple API pages worth of contributors (#2088) * Update contributors list (#2089) docs: update contributors list [skip ci] Co-authored-by: mrubens <[email protected]> * Refactor/cline.ts/list files (#2067) * "Refactor list_files tool to separate module (#2057)" * Await --------- Co-authored-by: Matt Rubens <[email protected]> * Remove switch from tools (#2091) * HandleError returns a promise * Remove unnecessary switch from tools * Fix a type in the listFiles tool (#2092) * Move write_to_file to a tool file (#2093) * Move apply_diff to a tool file (#2094) * Move insert_content to a tool file (#2095) * Move search_and_replace to a tool file (#2096) * Move list_code_definition_names to a tool file (#2097) * Move search_files to a tool file (#2098) * Move browser_action to a tool file (#2099) * Move execute_command to a tool file (#2100) * Move use_mcp_tool and access_mcp_resource to tool files (#2101) * Move new_task, switch_mode, attempt_completion, and ask_followup_question to tool files (#2102) * Do a little code cleanup now that tools are refactored out (#2103) * Clean up the way we compute the current diff strategy (#2049) * Clean up the way we compute the current diff strategy * Add changeset * i18n Add translations for task pinning and unpinning (#2109) Signed-off-by: feifei <[email protected]> * Remove the ask promise error (#2107) * Remove the ask promise error * Send start_time along with checkpoints and use it to position in the messages * Link to the settings page from the auto approve toolbar (#2111) * Link to provider docs from the API options (#2112) * Revert "Remove the ask promise error" (#2117) Revert "Remove the ask promise error (#2107)" This reverts commit 712ca71. * Parse the retry out of the gemini 429 response (#2118) * Fix Gemini command escaping (#2120) * Fix switching profiles to ensure only the selected profile is switche… (#2119) Fix switching profiles to ensure only the selected profile is switched for all modes, and optimize certain i18n keys. Signed-off-by: feifei <[email protected]> * Tweaks to support prompt line number format (#2121) * Remove the ask promise error (#2123) * Pass along pid and ppid * Revert "Remove the ask promise error" (#2126) Revert "Remove the ask promise error (#2123)" This reverts commit b788aa1. --------- Signed-off-by: feifei <[email protected]> Co-authored-by: Greg Taylor <[email protected]> Co-authored-by: Greg Taylor <[email protected]> Co-authored-by: Hannes Rudolph <[email protected]> Co-authored-by: Matt Rubens <[email protected]> Co-authored-by: aheizi <[email protected]> Co-authored-by: Afshawn Lotfi <[email protected]> Co-authored-by: Nico Bihan <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Bhavesh Ramburn <[email protected]> Co-authored-by: feifei <[email protected]>

Browser Automation Improvements

5bec372

* Added multi-tab remote Chrome support * Added support for hover * Properly caching remote browser host in global state * Cleanup functions

afshawnlotfi requested review from cte and mrubens as code owners March 12, 2025 18:18

github-project-automation bot added this to Roo Code Roadmap Mar 12, 2025

github-project-automation bot moved this to New in Roo Code Roadmap Mar 12, 2025

dosubot bot added the size:XL This PR changes 500-999 lines, ignoring generated files. label Mar 12, 2025

dosubot bot added the enhancement New feature or request label Mar 12, 2025

Merge branch 'main' into browser-improvements

aefd7d1

ellipsis-dev bot reviewed Mar 12, 2025

View reviewed changes

hannesrudolph moved this from New to PR [Pre Approval Review] in Roo Code Roadmap Mar 13, 2025

afshawnlotfi added 3 commits March 25, 2025 02:51

Updated for changes after merge

75017a3

Merge branch 'main' of https://github.com/RooVetGit/Roo-Code into bro…

d5bb8c7

…wser-improvements

Added www. exception for common tabs

748c4b7

dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. and removed size:XL This PR changes 500-999 lines, ignoring generated files. labels Mar 28, 2025

mrubens reviewed Mar 29, 2025

View reviewed changes

src/core/webview/ClineProvider.ts Outdated Show resolved Hide resolved

mrubens added 2 commits March 29, 2025 00:42

Update src/core/webview/ClineProvider.ts

fede7c8

Revert README changes

8810d67

dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:XXL This PR changes 1000+ lines, ignoring generated files. labels Mar 29, 2025

mrubens approved these changes Mar 29, 2025

View reviewed changes

dosubot bot added the lgtm This PR has been approved by a maintainer label Mar 29, 2025

mrubens merged commit cbe7075 into RooCodeInc:main Mar 29, 2025
13 checks passed

github-project-automation bot moved this from PR [Pre Approval Review] to Done in Roo Code Roadmap Mar 29, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

In-Editor Browser Improvements #1601

In-Editor Browser Improvements #1601

Uh oh!

afshawnlotfi commented Mar 12, 2025 •

edited by ellipsis-dev bot

Loading

Uh oh!

changeset-bot bot commented Mar 12, 2025 •

edited

Loading

Uh oh!

ellipsis-dev bot commented Mar 12, 2025

Uh oh!

ellipsis-dev bot Mar 12, 2025

Uh oh!

ellipsis-dev bot Mar 12, 2025

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

In-Editor Browser Improvements #1601

In-Editor Browser Improvements #1601

Uh oh!

Conversation

afshawnlotfi commented Mar 12, 2025 • edited by ellipsis-dev bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

New Features

Code Improvements

Bug Fixes

Testing

Uh oh!

changeset-bot bot commented Mar 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

ellipsis-dev bot commented Mar 12, 2025

Uh oh!

ellipsis-dev bot Mar 12, 2025

Choose a reason for hiding this comment

Uh oh!

ellipsis-dev bot Mar 12, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

afshawnlotfi commented Mar 12, 2025 •

edited by ellipsis-dev bot

Loading

changeset-bot bot commented Mar 12, 2025 •

edited

Loading