Web Crawling instead of "WebFetch"

[TomLucidor]

I have found the following article recommending MCPs that can be self-hosted https://archive.fo/JU21Q

• ArchiveBox and other WARC standard https://github.com/pragmar/mcp-server-webcrawl
• Crawl4AI https://github.com/coleam00/mcp-crawl4ai-rag
• FireCrawl https://github.com/firecrawl/firecrawl-mcp-server
• HyperBrowser https://github.com/hyperbrowserai/mcp
• SiteFetch (no systematic caching, just pure download) https://github.com/kentaro84207/mcp-sitefetch

Web Crawling is independent from search engines, especially when it comes to exploring blogs or full sites