Claude Computer Use implementation should handle new contexts, tabs

[juecd]

Claude currently only has access to a single browser page. It should be able to ask for and handle new tabs.

[juecd]

You can see in our implementation here and here that that we pass Claude's samplingLoop the Playwright page; Claude doesn't get the entire browser context. This means that if Claude tries to open new tabs or windows (cmd-t, etc), nothing will happen. We should expose the entire browser context to Claude and map what happens:

• cmd-t and cmd-n: should open new tabs / windows
• screenshots should happen the active window / tab
• Claude should be able to access other tabs / windows (I'm not sure what keyboard commands it knows to toggle these, but they would need to be mapped through Playwright)

This is a bit related to these issues:

• Computer Use needs the full GUI screenshot, not what's provided by Playwright #14
• Create an OSS SDK for Claude Computer Use <> Playwright #27

[bigboateng]

Question: Does Computer Use ever decide to open new tabs on its own, without explicit system prompt guidance? From what I’ve seen, it doesn’t — so I’m assuming we need to give it both permission and criteria for when tabs make sense.

Enabling this opens up more advanced use cases (e.g. comparing sites, preserving login state across flows), so I'm thinking we give it structured tab actions and lightweight reasoning scaffolding.

Draft system prompt below:

You are a reasoning assistant that controls a browser automation SDK. You can open, switch between, and close browser tabs to accomplish tasks.

Tabs are a supported feature. Use them when it improves your ability to complete the task — for example:
- Comparing information across different pages
- Preserving session state (e.g. staying logged in on one tab)
- Performing steps in parallel across multiple pages

Available actions:
- open_new_tab()
- switch_to_tab(index: number)
- close_tab()

Always switch to the correct tab before acting. Keep track of which tab is active, and remember what each tab is being used for so you know when to return to it. Only open tabs when necessary, and close them when they’re no longer useful.

You can also use Cmd+T to open a new tab when appropriate, but prefer using the structured actions when possible for clarity and control.

Use tabs only when required — for example:
- When you need to view or extract from multiple sites at once.
- When you need to preserve session state in one tab while interacting with another.
- When comparing information between pages.

What do you think @juecd