WebMCP integration

[domfarolino]

Hi! While working on Gemini-in-Chrome (which does not use a local model, but has the same kind of actuation capabilities as WebBrain), we and many partners found that actuation of web content can often be done much faster and much more accurately if the developer gets a chance to expose core site functionality to an agent in the form of "tools" it can understand.

Through this, we developed WebMCP which you may have heard of. Has there been any discussion of adding WebMCP support to WebBrain? This might be as easy as simply finding WebMCP tools through CDP and loading them into the model's context, to make it easier to actuate the site. I'd love to discuss a possible integration more, let me know what you think!

[esokullu]

This is really interesting! WebMCP could be a game-changer for browser agents like WebBrain.

The idea of exposing web app functionality as structured tools that agents can discover and invoke directly is compelling. Instead of relying solely on DOM manipulation and accessibility trees, we could have a more semantic layer where developers explicitly define what actions are available and how to invoke them.

A few thoughts:

Benefits:

• More reliable than DOM scraping - tools are explicitly defined with schemas
• Better performance - direct function calls vs. multiple DOM interactions
• Clearer intent - developers specify what operations make sense for agents
• Could reduce brittleness when UIs change

Questions to explore:

• How would tool discovery work? Would there be a standard manifest or registry?
• Security model - how do we prevent malicious sites from exposing harmful tools?
• Granularity - should tools be high-level (e.g., "create_issue") or low-level (e.g., "click_button")?
• How does this interact with existing approaches like accessibility trees?

I'd love to see a proof-of-concept integration. Even a simple example showing how WebBrain could discover and use WebMCP tools from a test site would be valuable.

@domfarolino have you thought about how this would work with the permission model? Users would need to understand what tools a site is exposing and consent to their use.

[domfarolino]

These are great questions, and we've been exploring all of them while developing WebMCP.

How would tool discovery work? Would there be a standard manifest or registry?

As of now, it's up to the agent to listen for toolchange events, which are fired when tools are registered and unregistered. We're considering integration with ARD, which would give a site the ability to declarative express the list of tool it supports (granularity and URL patterns/scope have not been discussed yet; we're working through the initial discussions). Declarative discovery is something we're very interested in though.

Security model - how do we prevent malicious sites from exposing harmful tools?

Unfortunately, this question is kind of like "how do we prevent malicious sites from exposing harmful forms to trick users". You probably can't, and it's becoming clear that defining a "harmful tool" and protecting agents from it is probably all part of the agent's security model, not WebMCP's security model. Another way to look at it is: even without WebMCP, the agent will encounter phishing sites that its security measures need to be robust against, and WebMCP probably isn't a novel threat vector. We have lots of discussion about this in https://webmachinelearning.github.io/webmcp/#security-privacy. There and elsewhere on the repository we discuss common security measures we're seeing agents employ, like spot-lighting, tool critics, agent origin sets, and many others. Feel free to contribute to the discussion—getting feedback from real-world agent users of WebMCP is super important to making it successful.

Granularity - should tools be high-level (e.g., "create_issue") or low-level (e.g., "click_button")?

We're running a live Origin Trial with partners right now to get a feel for how they best represent tools, to see what's successful and what's not. So far we're expecting them to be relatively high-level, and capture "user journeys" (i.e., "get me a refund on this flight") rather than just tool-ifying every possible UI interaction (i.e., "click button-a, then button-b, then button-c"), but exactly what successful patterns emerge is yet to be seen. The feedback we get from real-world site authors and agent vendors will help shape the API. My colleague @beaufortfrancois might be able to say more.

How does this interact with existing approaches like accessibility trees?

There's been some discussion about WebMCP <=> accessibility. The latest discussion is in webmachinelearning/webmcp#91 and in the explainer. We're open to feedback!

---

WebMCP is being shaped by feedback from both real-world agents and sites implementing tools, so please stay in touch; we'd love to hear any feedback!