feat: interactive browser session and supportsImages gating for browser_action

[hannesrudolph]

Related GitHub Issue

No tracking issue provided for this PR

Description

This PR adds an interactive browser session implementation and aligns browser tool availability with model image capability.

Concise summary of changes:

• New: BrowserSession
  • Local Chromium launch via puppeteer‑chromium‑resolver
  • Optional remote browser support (DevTools connect/disconnect) when enabled in state
  • Stable navigation (networkidle2 with timeout fallback to domcontentloaded)
  • Per‑action console log capture and PNG screenshot (base64 data URL)
  • Safe guards for test/mocked Page objects (only call setViewport, setExtraHTTPHeaders, mouse/keyboard, evaluate if present)
• Prompt gating updates:
  • generateSystemPrompt now gates interactive browser tool availability on model.info.supportsImages (instead of supportsComputerUse)
  • Task.getSystemPrompt uses the same supportsImages gating

What did not change:

• Existing runtime tool dispatch (browser_action via presentAssistantMessage/browserActionTool) was not altered functionally

Changes Made

• Added: src/services/browser/BrowserSession.ts
• Modified: src/core/webview/generateSystemPrompt.ts
• Modified: src/core/task/Task.ts
• Modified: src/services/browser/UrlContentFetcher.ts - add optional overrides for Chromium launch (chromiumArgs, chromiumExecutablePath)
• Modified: src/services/browser/tests/UrlContentFetcher.spec.ts - unit test for overrides

Test/CI

• Unit tests now pass on all platforms. The BrowserSession changes align with expectations in BrowserSession.spec.ts, including remote connect/disconnect behavior and guarded page method calls.
• CI status: all required checks green (analyze, compile, unit tests on ubuntu/windows, integration tests, knip).

Manual Verification

• Use a model with supportsImages and a mode that includes the “browser” group
• Invoke:
  • launch: <browser_action><action>launch</action><url>https://example.com</url></browser_action>
  • follow‑up actions (one per message): click/hover (with <coordinate>), type (<text>), scroll_up/scroll_down, resize (<size>)
  • close: <action>close</action>

Additional Notes

• No .gitignore or settings/UX changes included in this PR
• UrlContentFetcher now respects optional launch overrides from extension globalState:
  • chromiumArgs: string[] (appended to default args; duplicates removed)
  • chromiumExecutablePath: string (overrides puppeteer's resolved executablePath)
• Remote browser support depends on existing state keys (remoteBrowserEnabled, remoteBrowserHost); if enabled and reachable, DevTools connect is used, otherwise local Chromium is launched

[Copilot]

Pull Request Overview

This PR implements interactive browser session support via a new BrowserSession service and aligns browser tool gating to use the model's image capability (supportsImages) instead of computer use support.

• Rewrites BrowserSession class for simplified interactive browsing with local Chromium via puppeteer-chromium-resolver
• Changes browser tool gating from supportsComputerUse to supportsImages in both preview and runtime paths
• Removes complex remote browser connection logic in favor of streamlined local-only implementation

Reviewed Changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 4 comments.

File	Description
src/services/browser/BrowserSession.ts	Complete rewrite implementing simplified interactive browser session with console logging, screenshot capture, and basic navigation/interaction actions
src/core/webview/generateSystemPrompt.ts	Updates browser tool gating to check supportsImages instead of supportsComputerUse for model compatibility
src/core/task/Task.ts	Aligns runtime system prompt generation to use supportsImages for browser tool availability

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[roomote]

Thank you for this PR implementing interactive browser sessions! The refactoring from 560 to 280 lines is impressive and the switch from supportsComputerUse to supportsImages makes sense for screenshot-driven browsing. I've reviewed the changes and have some suggestions to improve type safety and code maintainability.