Tool suggestion: anybrowse for web scraping with Cloudflare bypass

[kc23go]

Problem with current web tools

smolagents VisitWebpageTool fails on Cloudflare-protected sites. For research agents this silently degrades quality -- empty content on a large fraction of useful URLs.

Alternative: anybrowse

anybrowse runs real residential Chrome and returns clean markdown. Works on Cloudflare-protected sites.

import requests
from smolagents import tool

@tool
def scrape_url(url: str) -> str:
    """Scrape any URL and return clean markdown content, including Cloudflare-protected sites."""
    r = requests.post("https://anybrowse.dev/scrape", json={"url": url})
    if r.status_code == 200:
        return r.json().get("markdown", "")
    return f"Failed to scrape: HTTP {r.status_code}"

Or via MCP:

{
  "mcpServers": {
    "anybrowse": {
      "type": "streamable-http",
      "url": "https://anybrowse.dev/mcp"
    }
  }
}

• Free: 10/day, no key
• Paid: $5 for 3,000 scrapes
• Docs: https://anybrowse.dev/docs