-
Notifications
You must be signed in to change notification settings - Fork 135
docs: update mcp.md #1583
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs: update mcp.md #1583
Changes from 29 commits
b03e5ca
be76c13
92217c7
be821af
a99eb47
1207230
672d703
4d3d304
36d3624
8a834b0
0340669
0ce013f
9540863
d061f36
f17955b
3ee3b25
9e7c370
75ef6f9
adcb648
9426d97
e78dcbd
56cf225
a10461f
11a5425
b374ff6
e3b68bf
a69a0c0
aea0a5c
52f0647
2e17507
b1c05b9
d6b5c77
d02475e
75d2d51
d758259
6310733
1ec3534
5fd1ee1
34d3a18
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -8,178 +8,124 @@ slug: /integrations/mcp | |||||||||||||||||||||
|
|
||||||||||||||||||||||
| <!-- markdownlint-disable MD024 --> | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| The Apify Model Context Protocol (MCP) server allows AI agents and frameworks compatible with the MCP standard to connect with the extensive library of Actors available on [Apify Store](https://apify.com/store). | ||||||||||||||||||||||
| The _Apify Model Context Protocol (MCP) Server_ allows AI applications to connect to Apify’s extensive library of Actors as tools to perform web scraping, data extraction, or other automation tasks in real time. | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| ## Why use MCP with Apify? | ||||||||||||||||||||||
|  | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| - _Access a vast tool library_: Provide AI agents access to thousands of pre-built Actors for web scraping, data extraction, and automation. | ||||||||||||||||||||||
| - _Dynamic tool discovery_: Unlike static OpenAPI definitions, MCP enables agents to dynamically find and add relevant Actors to their context during runtime. Agents can use MCP operations like `discover-actors` and `add-actor-to-tools`. | ||||||||||||||||||||||
| - _Scalability_: Efficiently manage access to a large and growing number of tools, which is challenging with single, static API definitions. | ||||||||||||||||||||||
| - _Flexible integration_: Connect to the MCP server using HTTP Server-Sent Events (SSE) or local standard input/output (stdin/stdout). Compatible clients include Claude Desktop, LibreChat, and [Apify’s Tester MCP Client](https://apify.com/jiri.spilka/tester-mcp-client). | ||||||||||||||||||||||
| ## Quickstart | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| ## Common use cases | ||||||||||||||||||||||
| You can use the Apify MCP Server in two ways: | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| - Standard Input/Output (stdio) which is useful for local integrations and command-line tools (e.g. Claude desktop) | ||||||||||||||||||||||
| - Use [mcp.apify.com](https://mcp.apify.com) by connecting your MCP client with `Authorization: Bearer <APIFY_TOKEN>` header | ||||||||||||||||||||||
vystrcild marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||||||||||||||||
|
|
||||||||||||||||||||||
| _(You could also use legacy option by running [Apify Actors MCP Server](https://apify.com/apify/actors-mcp-server) as Actor.)_ | ||||||||||||||||||||||
vystrcild marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||||||||||||||||
|
|
||||||||||||||||||||||
| - Extract data from websites, such as social media posts, search engine results, or specific URLs. | ||||||||||||||||||||||
| - Summarize web content or identify trends. | ||||||||||||||||||||||
| - Run automated web workflows without direct user interaction. | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| ## Prerequisites | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| Before you start, make sure you have the following: | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| 1. _Apify account_: You need an active account on the Apify platform. | ||||||||||||||||||||||
| 2. _API Token_: Get your personal API token from the **Integrations** section in [Apify Console](https://console.apify.com/account#/integrations). | ||||||||||||||||||||||
| 1. _An Apify account:_ Sign up for a free Apify account if you don’t have one. | ||||||||||||||||||||||
| 1. _Apify API Token:_ Get your personal API token from the **Integrations** section in [Apify Console](https://console.apify.com/account#/integrations). This token will be used to authorize the MCP server to run Actors on your behalf. | ||||||||||||||||||||||
| 1. _MCP client:_ You will also need an AI agent or client that supports MCP. This could be Anthropic Claude desktop, a VS Code extension with MCP support, Apify’s web-based Tester MCP Client, or any custom client implementation. See supported MCP clients in [official documentation](https://modelcontextprotocol.io/clients). | ||||||||||||||||||||||
vystrcild marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||||||||||||||||
|
|
||||||||||||||||||||||
| ## Connection methods | ||||||||||||||||||||||
| ## Example usage (local stdio with Claude desktop) | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| Apify provides two main ways to connect your MCP client to Actors: | ||||||||||||||||||||||
| Let’s walk through an example of using Claude desktop with the Apify MCP Server: | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| 1. Main Actors MCP Server: Access _all_ public Actors. Requires dynamic discovery or specifying Actors via URL parameters. | ||||||||||||||||||||||
| 2. Actor-specific MCP Server: A dedicated endpoint for a _single_ Actor, which is pre-registered and ready for immediate use. | ||||||||||||||||||||||
| 1. _Configure Claude desktop:_ Claude desktop supports MCP servers via the **Developer settings**. You need to add an entry for the Apify MCP server. For instance, in Claude’s config file, under `mcpServers`, add an entry like: | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| ### Option 1: Use the main Actors MCP server | ||||||||||||||||||||||
| ```json | ||||||||||||||||||||||
| { | ||||||||||||||||||||||
| "mcpServers": { | ||||||||||||||||||||||
| "actors-mcp-server": { | ||||||||||||||||||||||
| "command": "npx", | ||||||||||||||||||||||
| "args": ["-y", "@apify/actors-mcp-server"], | ||||||||||||||||||||||
| "env": { | ||||||||||||||||||||||
| "APIFY_TOKEN": "YOUR_APIFY_TOKEN" | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
| ``` | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| This method offers the most flexibility, allowing access to all of Apify Store dynamically or by specifying Actors at connection time. | ||||||||||||||||||||||
| This tells Claude to spawn the Apify MCP Server (via the [Actors MCP Server](https://www.npmjs.com/package/@apify/actors-mcp-server) NPM package with your API token (on the first run, it will download the package automatically). | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| ```text title="Server endpoint" | ||||||||||||||||||||||
| https://actors-mcp-server.apify.actor/sse?token=<YOUR_API_TOKEN> | ||||||||||||||||||||||
| ``` | ||||||||||||||||||||||
| 1. _Launch Claude and connect:_ After updating the config, restart Claude desktop. If successful, Claude will show a “plugin” (often indicated by a plug icon 🔌) signifying it connected to the Apify Actors MCP server. | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| Key features: | ||||||||||||||||||||||
| 1. _Use the Actors in conversation:_ You can chat with Claude and ask it to use Apify Actors. For example: _“What Apify Actors can I use?”_ Claude will list available tools via the MCP server. If none are pre-loaded, it may show defaults or guide you to find more. Then you can say: _“Use the Instagram Scraper to get NASA’s latest posts,”_ and Claude will run the apify/instagram-scraper Actor and return the results. | ||||||||||||||||||||||
vystrcild marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||||||||||||||||
|
|
||||||||||||||||||||||
| - Dynamic discovery: Agents can use MCP operations like `discover-actors` to search Apify Store for tools. | ||||||||||||||||||||||
| - Dynamic registration: Agents can use the `add-actor-to-tools` operation to add discovered Actors to their available toolset for the current session. | ||||||||||||||||||||||
| - Pre-registration via URL: You can make specific Actors available immediately by adding the `&actors=` query parameter to the connection URL (for example, `&actors=apify/rag-web-browser`). | ||||||||||||||||||||||
| _(If you prefer not to set up Claude desktop, you can achieve a similar result using [Apify’s Tester MCP Client](https://apify.com/jiri.spilka/tester-mcp-client), which provides a web UI to test the MCP server.)_ | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| The following example demonstrates connecting to the main server while pre-registering the `apify/rag-web-browser` Actor. | ||||||||||||||||||||||
|  | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| #### Step 1: Start the SSE connection | ||||||||||||||||||||||
| ## Interact with the MCP Server over SSE | ||||||||||||||||||||||
vystrcild marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||||||||||||||||
|
|
||||||||||||||||||||||
| Use `curl` or another SSE client to establish the connection. Append `&actors=apify/rag-web-browser` to pre-register the tool. Replace `<YOUR_API_TOKEN>` with your actual Apify API token. | ||||||||||||||||||||||
| You can interact with the server through Server-Sent Events (SSE) to send messages and receive responses. | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| ```bash | ||||||||||||||||||||||
| # Start the Server-Sent Events (SSE) session and keep it active | ||||||||||||||||||||||
| curl "https://actors-mcp-server.apify.actor/sse?token=<YOUR_API_TOKEN>&actors=apify/rag-web-browser" | ||||||||||||||||||||||
| In the client settings, you need to provide server configuration: | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| # The server responds with the session endpoint. Note the sessionId. | ||||||||||||||||||||||
| # event: endpoint | ||||||||||||||||||||||
| # data: /message?sessionId=9d820491-38d4-4c7d-bb6a-3b7dc542f1fa | ||||||||||||||||||||||
| ```json | ||||||||||||||||||||||
| { | ||||||||||||||||||||||
| "mcpServers": { | ||||||||||||||||||||||
| "apify": { | ||||||||||||||||||||||
| "type": "sse", | ||||||||||||||||||||||
| "url": "https://actors-mcp-server.apify.actor/sse", | ||||||||||||||||||||||
|
||||||||||||||||||||||
| "env": { | ||||||||||||||||||||||
vystrcild marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||||||||||||||||
| "APIFY_TOKEN": "your-apify-token" | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
| ``` | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| #### Step 2: Send a tool call request | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| Use the `sessionId` obtained in Step 1 to send a POST request to the `/message` endpoint. This request invokes the pre-registered `apify/rag-web-browser` tool. Replace `<YOUR_API_TOKEN>` and `<SESSION_ID>` with your actual values. | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| ```bash | ||||||||||||||||||||||
| curl -X POST "https://actors-mcp-server.apify.actor/message?token=<YOUR_API_TOKEN>&session_id=<SESSION_ID>" \ | ||||||||||||||||||||||
| -H "Content-Type: application/json" \ | ||||||||||||||||||||||
| -d '{ | ||||||||||||||||||||||
| "jsonrpc": "2.0", | ||||||||||||||||||||||
| "id": 1, | ||||||||||||||||||||||
| "method": "tools/call", | ||||||||||||||||||||||
| "params": { | ||||||||||||||||||||||
| "arguments": { | ||||||||||||||||||||||
| "query": "web browser for RAG pipelines -site:reddit.com", | ||||||||||||||||||||||
| "proxyConfiguration": { | ||||||||||||||||||||||
| "useApifyProxy": true | ||||||||||||||||||||||
| }, | ||||||||||||||||||||||
| "removeElementsCssSelector": "nav, footer, script, style, noscript, svg, img[src^='\''data:'\''],\\n[role=\\"alert\\"],\\n[role=\\"banner\\"],\\n[role=\\"dialog\\"],\\n[role=\\"alertdialog\\"],\\n[role=\\"region\\"][aria-label*=\\"skip\\" i],\\n[aria-modal=\\"true\\"]", | ||||||||||||||||||||||
| "htmlTransformer": "none" | ||||||||||||||||||||||
| }, | ||||||||||||||||||||||
| "name": "apify/rag-web-browser" | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
| }' | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| # The server immediately confirms receipt of the request: | ||||||||||||||||||||||
| # Accepted | ||||||||||||||||||||||
| ``` | ||||||||||||||||||||||
| ## Adding Multiple Actors | ||||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||
|
|
||||||||||||||||||||||
| #### Step 3: Receive the result via SSE | ||||||||||||||||||||||
| By default, the main Actors MCP Server starts with a single default Actor: [RAG Web Browser](https://apify.com/apify/rag-web-browser). However, you can fully customize which Actors are available: | ||||||||||||||||||||||
vystrcild marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||||||||||||||||
|
|
||||||||||||||||||||||
| The Actor's result is sent back asynchronously over the SSE connection established in Step 1. | ||||||||||||||||||||||
| - **Dynamic adding during a session:** If your client supports it, the agent itself can add Actors dynamically by name (using the `add-actor` operation) at runtime. For example, after using `search-actors` to find an Actor’s name, calling `add-actor` with that name will load it. Note that not all MCP client frameworks allow dynamic tool addition at runtime, but Apify’s own tester client does (when `enableActorAutoLoading` in Actors MCP Server is true). | ||||||||||||||||||||||
vystrcild marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
vystrcild marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||||||||||||||||
| - **Via config file (for Claude desktop):** When using Claude desktop, you can specify which Actors should be immediately available by configuring your `mcpServers` settings. Add the Actors as a comma-separated list in the `--actors` parameter, as shown in the example below. This pre-loads your selected tools without requiring discovery during conversations, ideal for workflows with predictable tool needs. | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| ```json | ||||||||||||||||||||||
| event: message | ||||||||||||||||||||||
| data: { | ||||||||||||||||||||||
| "result": { | ||||||||||||||||||||||
| "content": [ | ||||||||||||||||||||||
| { | ||||||||||||||||||||||
| "type": "text", | ||||||||||||||||||||||
| "text": "[{\"searchResult\":{\"title\":\"... RAG Web Browser Result ...\",\"description\":\"... Content extracted by the Actor ...\"}}]" // Example structure | ||||||||||||||||||||||
| { | ||||||||||||||||||||||
| "mcpServers": { | ||||||||||||||||||||||
| "actors-mcp-server": { | ||||||||||||||||||||||
| "command": "npx", | ||||||||||||||||||||||
| "args": [ | ||||||||||||||||||||||
| "-y", "@apify/actors-mcp-server", | ||||||||||||||||||||||
| "--actors", "lukaskrivka/google-maps-with-contact-details,apify/instagram-scraper" | ||||||||||||||||||||||
| ], | ||||||||||||||||||||||
| "env": { | ||||||||||||||||||||||
| "APIFY_TOKEN": "YOUR_APIFY_TOKEN" | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
| ] | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
| ``` | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| ### Option 2: Use an Actor-specific MCP server | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| Certain Actors, such as `apify/rag-web-browser`, offer their own dedicated MCP server endpoint. This simplifies integration when you only need to interact with that specific Actor. | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| ```text title="Endpoint example for `apify/rag-web-browser`" | ||||||||||||||||||||||
| https://rag-web-browser.apify.actor/sse?token=<YOUR_API_TOKEN> | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
| ``` | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| :::note | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| The hostname typically matches the Actor's name. | ||||||||||||||||||||||
| In summary, you can start with a broad set (everything open and discoverable) or a narrow set (just what you need) and even expand tools on the fly, giving your agent a lot of flexibility without overwhelming it initially. | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| ::: | ||||||||||||||||||||||
| ## Dynamic Actor Tooling | ||||||||||||||||||||||
vystrcild marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||||||||||||||||
|
|
||||||||||||||||||||||
| This method is ideal for integrating a specific tool directly into an application (like Claude Desktop) or a custom client without needing the dynamic discovery capabilities of the main server. | ||||||||||||||||||||||
| One of the powerful features of MCP with Apify is **dynamic actor tooling** – the ability for an AI agent to find new tools (Actors) as needed and incorporate them. Here are some special MCP operations and how Apify MCP Server supports them: | ||||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||
|
|
||||||||||||||||||||||
| The following example demonstrates use of `rag-web-browser` server. | ||||||||||||||||||||||
| - **`search-actors`:** Discover available Actors or MCP-Servers in Apify Store using full text search using keywords.Users try to discover Actors using free form query in this case search query must be converted to full text search. Returns a list of Actors with name, description, run statistics, pricing, starts, and URL. You perhaps need to use this tool several times to find the right Actor. You should prefer simple keywords over complex queries. Limit number of results returned but ensure that relevant results are returned. This is not a general search tool, it is designed to search for Actors in Apify Store. | ||||||||||||||||||||||
| - **`get-actor-details`:** Get documentation, readme, input schema and other details about an Actor. For example, when user says, I need to know more about web crawler Actor.Get details for an Actor with with Actor ID or Actor full name, i.e. username/name.Limit the length of the README if needed. | ||||||||||||||||||||||
| - **`help-tool`:** Helper tool to get information on how to use and troubleshoot the Apify MCP server. This tool always returns the same help message with information about the server and how to use it. Call this tool in case of any problems or uncertainties with the server. | ||||||||||||||||||||||
| - **`add-actor`:** Add a tool, Actor or MCP-Server to available tools by Actor ID or Actor name. A tool is an Actor or MCP-Server that can be called by the userDo not execute the tool, only add it and list it in available tools. For example, add a tool with username/name when user wants to scrape data from a website. | ||||||||||||||||||||||
vystrcild marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||||||||||||||||
| - **`remove-actor`:** Remove a tool, an Actor or MCP-Server by name from available tools. For example, when user says, I do not need a tool username/name anymore | ||||||||||||||||||||||
|
||||||||||||||||||||||
| - **`search-actors`:** Discover available Actors or MCP-Servers in Apify Store using full text search using keywords.Users try to discover Actors using free form query in this case search query must be converted to full text search. Returns a list of Actors with name, description, run statistics, pricing, starts, and URL. You perhaps need to use this tool several times to find the right Actor. You should prefer simple keywords over complex queries. Limit number of results returned but ensure that relevant results are returned. This is not a general search tool, it is designed to search for Actors in Apify Store. | |
| - **`get-actor-details`:** Get documentation, readme, input schema and other details about an Actor. For example, when user says, I need to know more about web crawler Actor.Get details for an Actor with with Actor ID or Actor full name, i.e. username/name.Limit the length of the README if needed. | |
| - **`help-tool`:** Helper tool to get information on how to use and troubleshoot the Apify MCP server. This tool always returns the same help message with information about the server and how to use it. Call this tool in case of any problems or uncertainties with the server. | |
| - **`add-actor`:** Add a tool, Actor or MCP-Server to available tools by Actor ID or Actor name. A tool is an Actor or MCP-Server that can be called by the userDo not execute the tool, only add it and list it in available tools. For example, add a tool with username/name when user wants to scrape data from a website. | |
| - **`remove-actor`:** Remove a tool, an Actor or MCP-Server by name from available tools. For example, when user says, I do not need a tool username/name anymore | |
| - `search-actors`: Discover available Actors or MCP-Servers in Apify Store using full text search using keywords.Users try to discover Actors using free form query in this case search query must be converted to full text search. Returns a list of Actors with name, description, run statistics, pricing, starts, and URL. You perhaps need to use this tool several times to find the right Actor. You should prefer simple keywords over complex queries. Limit number of results returned but ensure that relevant results are returned. This is not a general search tool, it is designed to search for Actors in Apify Store. | |
| - `get-actor-details`: Get documentation, readme, input schema and other details about an Actor. For example, when user says, I need to know more about web crawler Actor.Get details for an Actor with with Actor ID or Actor full name, i.e. username/name.Limit the length of the README if needed. | |
| - `help-tool`: Helper tool to get information on how to use and troubleshoot the Apify MCP server. This tool always returns the same help message with information about the server and how to use it. Call this tool in case of any problems or uncertainties with the server. | |
| - `add-actor`: Add a tool, Actor or MCP-Server to available tools by Actor ID or Actor name. A tool is an Actor or MCP-Server that can be called by the userDo not execute the tool, only add it and list it in available tools. For example, add a tool with username/name when user wants to scrape data from a website. | |
| - `remove-actor`: Remove a tool, an Actor or MCP-Server by name from available tools. For example, when user says, I do not need a tool username/name anymore |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we really need to have this list? And if so, do we need it so long?
Uh oh!
There was an error while loading. Please reload this page.