diff --git a/README.md b/README.md index 50cd3e27..b1e8a455 100644 --- a/README.md +++ b/README.md @@ -140,19 +140,10 @@ One of the most powerful features of using MCP with Apify is dynamic tool discov It gives an AI agent the ability to find new tools (Actors) as needed and incorporate them. Here are some special MCP operations and how the Apify MCP Server supports them: -- **Actor discovery and management**: Search for Actors, view their details, and dynamically add or remove them as available tools for the AI. +- **Apify Actors**: Search for Actors, view their details, and use them as tools for the AI. - **Apify documentation**: Search the Apify documentation and fetch specific documents to provide context to the AI. -- **Actor runs (*)**: Get lists of your Actor runs, inspect their details, and retrieve logs. -- **Apify storage (*)**: Access data from your datasets and key-value stores. - -**Note**: Helper tool categories marked with (*) are not enabled by default in the MCP server and must be explicitly enabled using the `tools` argument (either the `--tools` command line argument for the stdio server or the `?tools` URL query parameter for the remote MCP server). The `tools` argument is a comma-separated list of categories with the following possible values: - -- `docs`: Search and fetch Apify documentation tools. -- `runs`: Get Actor run lists, run details, and logs from a specific Actor run. -- `storage`: Access datasets, key-value stores, and their records. -- `preview`: Experimental tools in preview mode. - -For example, to enable all tools, use `npx @apify/actors-mcp-server --tools docs,runs,storage,preview` or `https://mcp.apify.com/?tools=docs,runs,storage,preview`. +- **Actor runs**: Get lists of your Actor runs, inspect their details, and retrieve logs. +- **Apify storage**: Access data from your datasets and key-value stores. ### Overview of available tools @@ -160,23 +151,95 @@ Here is an overview list of all the tools provided by the Apify MCP Server. | Tool name | Category | Description | Enabled by default | | :--- | :--- | :--- | :---: | -| `get-actor-details` | default | Retrieve detailed information about a specific Actor. | ✅ | -| `search-actors` | default | Search for Actors in the Apify Store. | ✅ | -| `add-actor` | default | Add an Actor as a new tool for the user to call. | ✅ | -| [`apify-slash-rag-web-browser`](https://apify.com/apify/rag-web-browser) | default | An Actor tool to browse the web. | ✅ | +| `search-actors` | actors | Search for Actors in the Apify Store. | ✅ | +| `fetch-actor-details` | actors | Retrieve detailed information about a specific Actor. | ✅ | +| `call-actor` | actors | Call an Actor and get its run results. | ✅ | +| [`apify-slash-rag-web-browser`](https://apify.com/apify/rag-web-browser) | Actor (see [tool configuration](#tools-configuration)) | An Actor tool to browse the web. | ✅ | | `search-apify-docs` | docs | Search the Apify documentation for relevant pages. | ✅ | | `fetch-apify-docs` | docs | Fetch the full content of an Apify documentation page by its URL. | ✅ | -| `call-actor` | preview | Call an Actor and get its run results. | | | `get-actor-run` | runs | Get detailed information about a specific Actor run. | | | `get-actor-run-list` | runs | Get a list of an Actor's runs, filterable by status. | | | `get-actor-log` | runs | Retrieve the logs for a specific Actor run. | | | `get-dataset` | storage | Get metadata about a specific dataset. | | | `get-dataset-items` | storage | Retrieve items from a dataset with support for filtering and pagination. | | +| `get-dataset-schema` | storage | Generate a JSON schema from dataset items. | | | `get-key-value-store` | storage | Get metadata about a specific key-value store. | | | `get-key-value-store-keys`| storage | List the keys within a specific key-value store. | | | `get-key-value-store-record`| storage | Get the value associated with a specific key in a key-value store. | | | `get-dataset-list` | storage | List all available datasets for the user. | | | `get-key-value-store-list`| storage | List all available key-value stores for the user. | | +| `add-actor` | experimental | Add an Actor as a new tool for the user to call. | | + +### Tools configuration + +The `tools` configuration parameter is used to specify loaded tools - either categories or specific tools directly, and Apify Actors. For example, `tools=storage,runs` loads two categories; `tools=add-actor` loads just one tool. + +When no query parameters are provided, the MCP server loads the following `tools` by default: + +- `actors` +- `docs` +- `apify/rag-web-browser` + +If the tools parameter is specified, only the listed tools or categories will be enabled - no default tools will be included. + +> **Easy configuration:** +> +> Use the [UI configurator](https://mcp.apify.com/) to configure your server, then copy the configuration to your client. + +**Configuring the hosted server:** + +The hosted server can be configured using query parameters in the URL. For example, to load the default tools, use: + +``` +https://mcp.apify.com?tools=actors,docs,apify/rag-web-browser +``` + +For minimal configuration, if you want to use only a single Actor tool - without any discovery or generic calling tools, the server can be configured as follows: + +``` +https://mcp.apify.com?tools=apify/my-actor +``` + +This setup exposes only the specified Actor (`apify/my-actor`) as a tool. No other tools will be available. + +**Configuring the CLI:** + +The CLI can be configured using command-line flags. For example, to load the same tools as in the hosted server configuration, use: + +```bash +npx @apify/actors-mcp-server --tools actors,docs,apify/rag-web-browser +``` + +The minimal configuration is similar to the hosted server configuration: + +```bash +npx @apify/actors-mcp-server --tools apify/my-actor +``` + +As above, this exposes only the specified Actor (`apify/my-actor`) as a tool. No other tools will be available. + +> **⚠️ Important recommendation** +> +> **The default tools configuration may change in future versions.** When no `tools` parameter is specified, the server currently loads default tools, but this behavior is subject to change. +> +> **For production use and stable interfaces, always explicitly specify the `tools` parameter** to ensure your configuration remains consistent across updates. + +### Backward compatibility + +The v2 configuration preserves backward compatibility with v1 usage. Notes: + +- `actors` param (URL) and `--actors` flag (CLI) are still supported. + - Internally they are merged into `tools` selectors. + - Examples: `?actors=apify/rag-web-browser` ≡ `?tools=apify/rag-web-browser`; `--actors apify/rag-web-browser` ≡ `--tools apify/rag-web-browser`. +- `enable-adding-actors` (CLI) and `enableAddingActors` (URL) are supported but deprecated. + - Prefer `tools=experimental` or including the specific tool `tools=add-actor`. + - Behavior remains: when enabled with no `tools` specified, the server exposes only `add-actor`; when categories/tools are selected, `add-actor` is also included. +- `enableActorAutoLoading` remains as a legacy alias for `enableAddingActors` and is mapped automatically. +- Defaults remain compatible: when no `tools` are specified, the server loads `actors`, `docs`, and `apify/rag-web-browser`. + - If any `tools` are specified, the defaults are not added (same as v1 intent for explicit selection). +- `call-actor` is now included by default via the `actors` category (additive change). To exclude it, specify an explicit `tools` list without `actors`. + +Existing URLs and commands using `?actors=...` or `--actors` continue to work unchanged. ### Prompts diff --git a/manifest.json b/manifest.json index 34f9cce3..3ca704df 100644 --- a/manifest.json +++ b/manifest.json @@ -1,7 +1,7 @@ { "dxt_version": "0.1", "name": "apify-mcp-server", - "version": "0.3.9", + "version": "0.4.0", "description": "Extract data from any site with Apify Store, home to thousands of web scrapers.", "long_description": "Apify is the world's largest marketplace of tools for web scraping, data extraction, and web automation. You can extract structured data from social media, e-commerce, search engines, maps, travel sites, or any other website.", "keywords": [ @@ -42,11 +42,7 @@ "args": [ "${__dirname}/dist/stdio.js", "--tools", - "${user_config.tools}", - "--actors", - "${user_config.actors}", - "--enable-adding-actors", - "${user_config.enable-adding-actors}" + "${user_config.tools}" ], "env": { "APIFY_TOKEN": "${user_config.apify_token}" @@ -64,23 +60,10 @@ }, "tools": { "type": "string", - "title": "Enabled tool categories", - "description": "A comma-separated list of tool categories to enable. Available options: docs, runs, storage, preview.", + "title": "Enabled tools", + "description": "Comma-separated list of tools to enable. Can be either a tool category, a specific tool, or an Apify Actor. For example: \"actors,docs,apify/rag-web-browser\". For more details visit https://mcp.apify.com.", "required": false, - "default": "docs" - }, - "actors": { - "type": "string", - "title": "Enabled Actors", - "description": "A comma-separated list of full Actor names to add to the server on startup (e.g., apify/rag-web-browser).", - "required": false, - "default": "apify/rag-web-browser" - }, - "enable-adding-actors": { - "type": "boolean", - "title": "Enable dynamic Actor adding", - "description": "Allow dynamically adding Actors as tools based on user requests during a session.", - "default": true + "default": "actors,docs,apify/rag-web-browser" } }, "compatibility": { diff --git a/src/actor/server.ts b/src/actor/server.ts index db449608..b1c52248 100644 --- a/src/actor/server.ts +++ b/src/actor/server.ts @@ -12,17 +12,11 @@ import express from 'express'; import log from '@apify/log'; import { ActorsMcpServer } from '../mcp/server.js'; -import { parseInputParamsFromUrl } from '../mcp/utils.js'; import { getHelpMessage, HEADER_READINESS_PROBE, Routes, TransportType } from './const.js'; import { getActorRunData } from './utils.js'; export function createExpressApp( host: string, - mcpServerOptions: { - enableAddingActors?: boolean; - enableDefaultActors?: boolean; - actors?: string[]; - }, ): express.Express { const app = express(); const mcpServers: { [sessionId: string]: ActorsMcpServer } = {}; @@ -75,21 +69,13 @@ export function createExpressApp( rt: Routes.SSE, tr: TransportType.SSE, }); - const mcpServer = new ActorsMcpServer(mcpServerOptions, false); + const mcpServer = new ActorsMcpServer(false); const transport = new SSEServerTransport(Routes.MESSAGE, res); // Load MCP server tools const apifyToken = process.env.APIFY_TOKEN as string; - const input = parseInputParamsFromUrl(req.url); - if (input.actors || input.enableAddingActors || input.tools) { - log.debug('Loading tools from URL', { sessionId: transport.sessionId, tr: TransportType.SSE }); - await mcpServer.loadToolsFromUrl(req.url, apifyToken); - } - // Load default tools if no actors are specified - if (!input.actors) { - log.debug('Loading default tools', { sessionId: transport.sessionId, tr: TransportType.SSE }); - await mcpServer.loadDefaultActors(apifyToken); - } + log.debug('Loading tools from URL', { sessionId: transport.sessionId, tr: TransportType.SSE }); + await mcpServer.loadToolsFromUrl(req.url, apifyToken); transportsSSE[transport.sessionId] = transport; mcpServers[transport.sessionId] = mcpServer; @@ -166,20 +152,12 @@ export function createExpressApp( sessionIdGenerator: () => randomUUID(), enableJsonResponse: false, // Use SSE response mode }); - const mcpServer = new ActorsMcpServer(mcpServerOptions, false); + const mcpServer = new ActorsMcpServer(false); // Load MCP server tools const apifyToken = process.env.APIFY_TOKEN as string; - const input = parseInputParamsFromUrl(req.url); - if (input.actors || input.enableAddingActors || input.tools) { - log.debug('Loading tools from URL', { sessionId: transport.sessionId, tr: TransportType.HTTP }); - await mcpServer.loadToolsFromUrl(req.url, apifyToken); - } - // Load default tools if no actors are specified - if (!input.actors) { - log.debug('Loading default tools', { sessionId: transport.sessionId, tr: TransportType.HTTP }); - await mcpServer.loadDefaultActors(apifyToken); - } + log.debug('Loading tools from URL', { sessionId: transport.sessionId, tr: TransportType.HTTP }); + await mcpServer.loadToolsFromUrl(req.url, apifyToken); // Connect the transport to the MCP server BEFORE handling the request await mcpServer.connect(transport); diff --git a/src/const.ts b/src/const.ts index 2e6797c9..ff819fdf 100644 --- a/src/const.ts +++ b/src/const.ts @@ -19,7 +19,7 @@ export enum HelperTools { ACTOR_ADD = 'add-actor', ACTOR_CALL = 'call-actor', ACTOR_GET = 'get-actor', - ACTOR_GET_DETAILS = 'get-actor-details', + ACTOR_GET_DETAILS = 'fetch-actor-details', ACTOR_REMOVE = 'remove-actor', ACTOR_RUNS_ABORT = 'abort-actor-run', ACTOR_RUNS_GET = 'get-actor-run', diff --git a/src/examples/clientSse.ts b/src/examples/clientSse.ts deleted file mode 100644 index d198af60..00000000 --- a/src/examples/clientSse.ts +++ /dev/null @@ -1,120 +0,0 @@ -/* eslint-disable no-console */ -/** - * Connect to the MCP server using SSE transport and call a tool. - * The Apify MCP Server will load default Actors. - * - * It requires the `APIFY_TOKEN` in the `.env` file. - */ - -import path from 'node:path'; -import { fileURLToPath } from 'node:url'; - -import { Client } from '@modelcontextprotocol/sdk/client/index.js'; -import { SSEClientTransport } from '@modelcontextprotocol/sdk/client/sse.js'; -import { CallToolResultSchema } from '@modelcontextprotocol/sdk/types.js'; -import dotenv from 'dotenv'; // eslint-disable-line import/no-extraneous-dependencies -import type { EventSourceInit } from 'eventsource'; -import { EventSource } from 'eventsource'; // eslint-disable-line import/no-extraneous-dependencies - -import { actorNameToToolName } from '../tools/utils.js'; - -const REQUEST_TIMEOUT = 120_000; // 2 minutes -const filename = fileURLToPath(import.meta.url); -const dirname = path.dirname(filename); - -dotenv.config({ path: path.resolve(dirname, '../../.env') }); - -const SERVER_URL = process.env.MCP_SERVER_URL_BASE || 'https://actors-mcp-server.apify.actor/sse'; -// We need to change forward slash / to underscore -- in the tool name as Anthropic does not allow forward slashes in the tool name -const SELECTED_TOOL = actorNameToToolName('apify/rag-web-browser'); -// const QUERY = 'web browser for Anthropic'; -const QUERY = 'apify'; - -if (!process.env.APIFY_TOKEN) { - console.error('APIFY_TOKEN is required but not set in the environment variables.'); - process.exit(1); -} - -// Declare EventSource on globalThis if not available (needed for Node.js environment) -declare global { - - // eslint-disable-next-line no-var, vars-on-top - var EventSource: { - new(url: string, eventSourceInitDict?: EventSourceInit): EventSource; - prototype: EventSource; - CONNECTING: 0; - OPEN: 1; - CLOSED: 2; - }; -} - -if (typeof globalThis.EventSource === 'undefined') { - globalThis.EventSource = EventSource; -} - -async function main(): Promise { - const transport = new SSEClientTransport( - new URL(SERVER_URL), - { - requestInit: { - headers: { - authorization: `Bearer ${process.env.APIFY_TOKEN}`, - }, - }, - eventSourceInit: { - // The EventSource package augments EventSourceInit with a "fetch" parameter. - // You can use this to set additional headers on the outgoing request. - // Based on this example: https://github.com/modelcontextprotocol/typescript-sdk/issues/118 - async fetch(input: Request | URL | string, init?: RequestInit) { - const headers = new Headers(init?.headers || {}); - headers.set('authorization', `Bearer ${process.env.APIFY_TOKEN}`); - return fetch(input, { ...init, headers }); - }, - // We have to cast to "any" to use it, since it's non-standard - } as any, // eslint-disable-line @typescript-eslint/no-explicit-any - }, - ); - const client = new Client( - { name: 'example-client', version: '1.0.0' }, - { capabilities: {} }, - ); - - try { - // Connect to the MCP server - await client.connect(transport); - - // List available tools - const tools = await client.listTools(); - console.log('Available tools:', tools); - - if (tools.tools.length === 0) { - console.log('No tools available'); - return; - } - - const selectedTool = tools.tools.find((tool) => tool.name === SELECTED_TOOL); - if (!selectedTool) { - console.error(`The specified tool: ${selectedTool} is not available. Exiting.`); - return; - } - - // Call a tool - console.log(`Calling actor ... ${SELECTED_TOOL}`); - const result = await client.callTool( - { name: SELECTED_TOOL, arguments: { query: QUERY } }, - CallToolResultSchema, - { timeout: REQUEST_TIMEOUT }, - ); - console.log('Tool result:', JSON.stringify(result, null, 2)); - } catch (error: unknown) { - if (error instanceof Error) { - console.error('Error:', error.message); - } else { - console.error('An unknown error occurred:', error); - } - } finally { - await client.close(); - } -} - -await main(); diff --git a/src/examples/clientStdio.ts b/src/examples/clientStdio.ts deleted file mode 100644 index bb072f6d..00000000 --- a/src/examples/clientStdio.ts +++ /dev/null @@ -1,89 +0,0 @@ -/* eslint-disable no-console */ -/** - * Connect to the MCP server using stdio transport and call a tool. - * This script uses a selected tool without LLM involvement. - * You need to provide the path to the MCP server and `APIFY_TOKEN` in the `.env` file. - * You can choose actors to run in the server, for example: `apify/rag-web-browser`. - */ - -import { execSync } from 'node:child_process'; -import path from 'node:path'; -import { fileURLToPath } from 'node:url'; - -import { Client } from '@modelcontextprotocol/sdk/client/index.js'; -import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js'; -import { CallToolResultSchema } from '@modelcontextprotocol/sdk/types.js'; -import dotenv from 'dotenv'; // eslint-disable-line import/no-extraneous-dependencies - -import { actorNameToToolName } from '../tools/utils.js'; - -// Resolve dirname equivalent in ES module -const filename = fileURLToPath(import.meta.url); -const dirname = path.dirname(filename); - -dotenv.config({ path: path.resolve(dirname, '../../.env') }); -const SERVER_PATH = path.resolve(dirname, '../../dist/stdio.js'); -const NODE_PATH = execSync(process.platform === 'win32' ? 'where node' : 'which node').toString().trim(); - -const TOOLS = 'apify/rag-web-browser,lukaskrivka/google-maps-with-contact-details'; -const SELECTED_TOOL = actorNameToToolName('apify/rag-web-browser'); - -if (!process.env.APIFY_TOKEN) { - console.error('APIFY_TOKEN is required but not set in the environment variables.'); - process.exit(1); -} - -// Create server parameters for stdio connection -const transport = new StdioClientTransport({ - command: NODE_PATH, - args: [SERVER_PATH, '--actors', TOOLS], - env: { APIFY_TOKEN: process.env.APIFY_TOKEN || '' }, -}); - -// Create a new client instance -const client = new Client( - { name: 'example-client', version: '0.1.0' }, - { capabilities: {} }, -); - -// Main function to run the example client -async function run() { - try { - // Connect to the MCP server - await client.connect(transport); - - // List available tools - const tools = await client.listTools(); - console.log('Available tools:', tools); - - if (tools.tools.length === 0) { - console.log('No tools available'); - return; - } - - // Example: Call the first available tool - const selectedTool = tools.tools.find((tool) => tool.name === SELECTED_TOOL); - - if (!selectedTool) { - console.error(`The specified tool: ${selectedTool} is not available. Exiting.`); - return; - } - - // Call a tool - console.log('Calling actor ...'); - const result = await client.callTool( - { name: SELECTED_TOOL, arguments: { query: 'web browser for Anthropic' } }, - CallToolResultSchema, - ); - console.log('Tool result:', JSON.stringify(result)); - - await client.close(); - } catch (error) { - console.error('Error:', error); - } -} - -run().catch((error) => { - console.error(`Error running MCP client: ${error as Error}`); - process.exit(1); -}); diff --git a/src/examples/clientStdioChat.ts b/src/examples/clientStdioChat.ts deleted file mode 100644 index bd1118c9..00000000 --- a/src/examples/clientStdioChat.ts +++ /dev/null @@ -1,227 +0,0 @@ -/* eslint-disable no-console */ -/** - * Create a simple chat client that connects to the Model Context Protocol server using the stdio transport. - * Based on the user input, the client sends a query to the MCP server, retrieves results and processes them. - * - * You can expect the following output: - * - * MCP Client Started! - * Type your queries or 'quit|q|exit' to exit. - * You: Find to articles about AI agent and return URLs - * [internal] Received response from Claude: [{"type":"text","text":"I'll search for information about AI agents - * and provide you with a summary."},{"type":"tool_use","id":"tool_01He9TkzQfh2979bbeuxWVqM","name":"search", - * "input":{"query":"what are AI agents definition capabilities applications","maxResults":2}}] - * [internal] Calling tool: {"name":"search","arguments":{"query":"what are AI agents definition ... - * I can help analyze the provided content about AI agents. - * This appears to be crawled content from AWS and IBM websites explaining what AI agents are. - * Let me summarize the key points: - */ - -import { execSync } from 'node:child_process'; -import path from 'node:path'; -import * as readline from 'node:readline'; -import { fileURLToPath } from 'node:url'; - -import { Anthropic } from '@anthropic-ai/sdk'; // eslint-disable-line import/no-extraneous-dependencies -import type { Message, MessageParam, ToolUseBlock } from '@anthropic-ai/sdk/resources/messages'; -import { Client } from '@modelcontextprotocol/sdk/client/index.js'; -import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js'; -import { CallToolResultSchema } from '@modelcontextprotocol/sdk/types.js'; -import dotenv from 'dotenv'; // eslint-disable-line import/no-extraneous-dependencies - -const filename = fileURLToPath(import.meta.url); -const dirname = path.dirname(filename); - -dotenv.config({ path: path.resolve(dirname, '../../.env') }); - -const REQUEST_TIMEOUT = 120_000; // 2 minutes -const MAX_TOKENS = 2048; // Maximum tokens for Claude response - -// const CLAUDE_MODEL = 'claude-3-5-sonnet-20241022'; // the most intelligent model -// const CLAUDE_MODEL = 'claude-3-5-haiku-20241022'; // a fastest model -const CLAUDE_MODEL = 'claude-3-haiku-20240307'; // a fastest and most compact model for near-instant responsiveness -const DEBUG = true; -const DEBUG_SERVER_PATH = path.resolve(dirname, '../../dist/stdio.js'); - -const NODE_PATH = execSync('which node').toString().trim(); - -dotenv.config(); // Load environment variables from .env - -export type Tool = { - name: string; - description: string | undefined; - input_schema: unknown; -} - -class MCPClient { - private anthropic: Anthropic; - private client = new Client( - { - name: 'example-client', - version: '0.1.0', - }, - { - capabilities: {}, // Optional capabilities - }, - ); - - private tools: Tool[] = []; - - constructor() { - this.anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY }); - } - - /** - * Start the server using node and provided server script path. - * Connect to the server using stdio transport and list available tools. - */ - async connectToServer(serverArgs: string[]) { - const transport = new StdioClientTransport({ - command: NODE_PATH, - args: serverArgs, - env: { APIFY_TOKEN: process.env.APIFY_TOKEN || '' }, - }); - - await this.client.connect(transport); - const response = await this.client.listTools(); - - this.tools = response.tools.map((x) => ({ - name: x.name, - description: x.description, - input_schema: x.inputSchema, - })); - console.log('Connected to server with tools:', this.tools.map((x) => x.name)); - } - - /** - * Process LLM response and check whether it contains any tool calls. - * If a tool call is found, call the tool and return the response and save the results to messages with type: user. - * If the tools response is too large, truncate it to the limit. - */ - async processMsg(response: Message, messages: MessageParam[]): Promise { - for (const content of response.content) { - if (content.type === 'text') { - messages.push({ role: 'assistant', content: content.text }); - } else if (content.type === 'tool_use') { - await this.handleToolCall(content, messages); - } - } - return messages; - } - - /** - * Call the tool and return the response. - */ - private async handleToolCall(content: ToolUseBlock, messages: MessageParam[], toolCallCount = 0): Promise { - // eslint-disable-next-line @typescript-eslint/no-explicit-any - const params = { name: content.name, arguments: content.input as any }; - console.log(`[internal] Calling tool (count: ${toolCallCount}): ${JSON.stringify(params)}`); - let results; - try { - results = await this.client.callTool(params, CallToolResultSchema, { timeout: REQUEST_TIMEOUT }); - if (results.content instanceof Array && results.content.length !== 0) { - const text = results.content.map((x) => x.text); - messages.push({ role: 'user', content: `Tool result: ${text.join('\n\n')}` }); - } else { - messages.push({ role: 'user', content: `No results retrieved from ${params.name}` }); - } - } catch (error) { - messages.push({ role: 'user', content: `Error calling tool: ${params.name}, error: ${error}` }); - } - // Get next response from Claude - const nextResponse: Message = await this.anthropic.messages.create({ - model: CLAUDE_MODEL, - max_tokens: MAX_TOKENS, - messages, - tools: this.tools as any[], // eslint-disable-line @typescript-eslint/no-explicit-any - }); - - for (const c of nextResponse.content) { - if (c.type === 'text') { - messages.push({ role: 'assistant', content: c.text }); - } else if (c.type === 'tool_use' && toolCallCount < 3) { - return await this.handleToolCall(c, messages, toolCallCount + 1); - } - } - - return messages; - } - - /** - * Process user query by sending it to the server and returning the response. - * Also, process any tool calls. - */ - async processQuery(query: string, messages: MessageParam[]): Promise { - messages.push({ role: 'user', content: query }); - const response: Message = await this.anthropic.messages.create({ - model: CLAUDE_MODEL, - max_tokens: MAX_TOKENS, - messages, - tools: this.tools as any[], // eslint-disable-line @typescript-eslint/no-explicit-any - }); - console.log('[internal] Received response from Claude:', JSON.stringify(response.content)); - return await this.processMsg(response, messages); - } - - /** - * Create a chat loop that reads user input from the console and sends it to the server for processing. - */ - async chatLoop() { - const rl = readline.createInterface({ - input: process.stdin, - output: process.stdout, - prompt: 'You: ', - }); - - console.log("MCP Client Started!\nType your queries or 'quit|q|exit' to exit."); - rl.prompt(); - - let lastPrintMessage = 0; - const messages: MessageParam[] = []; - rl.on('line', async (input) => { - const v = input.trim().toLowerCase(); - if (v === 'quit' || v === 'q' || v === 'exit') { - rl.close(); - return; - } - try { - await this.processQuery(input, messages); - for (let i = lastPrintMessage + 1; i < messages.length; i++) { - if (messages[i].role === 'assistant') { - console.log('CLAUDE:', messages[i].content); - } else if (messages[i].role === 'user') { - console.log('USER:', messages[i].content.slice(0, 500), '...'); - } else { - console.log('CLAUDE[thinking]:', messages[i].content); - } - } - lastPrintMessage += messages.length; - } catch (error) { - console.error('Error processing query:', error); - } - rl.prompt(); - }); - } -} - -async function main() { - const client = new MCPClient(); - - if (process.argv.length < 3) { - if (DEBUG) { - process.argv.push(DEBUG_SERVER_PATH); - } else { - console.error('Usage: node '); - process.exit(1); - } - } - - try { - await client.connectToServer(process.argv.slice(2)); - await client.chatLoop(); - } catch (error) { - console.error('Error:', error); - } -} - -main().catch(console.error); diff --git a/src/examples/clientStreamableHttp.ts b/src/examples/clientStreamableHttp.ts deleted file mode 100644 index d62dcbe1..00000000 --- a/src/examples/clientStreamableHttp.ts +++ /dev/null @@ -1,113 +0,0 @@ -import { Client } from '@modelcontextprotocol/sdk/client/index.js'; -import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamableHttp.js'; -import type { CallToolRequest, ListToolsRequest } from '@modelcontextprotocol/sdk/types.js'; -import { - CallToolResultSchema, - ListToolsResultSchema, - LoggingMessageNotificationSchema, -} from '@modelcontextprotocol/sdk/types.js'; - -import log from '@apify/log'; - -import { HelperTools } from '../const.js'; - -log.setLevel(log.LEVELS.DEBUG); - -async function main(): Promise { - // Create a new client with streamable HTTP transport - const client = new Client({ - name: 'example-client', - version: '1.0.0', - }); - - const transport = new StreamableHTTPClientTransport( - new URL('http://localhost:3000/mcp'), - ); - - // Connect the client using the transport and initialize the server - await client.connect(transport); - log.debug('Connected to MCP server'); - - // Set up notification handlers for server-initiated messages - client.setNotificationHandler(LoggingMessageNotificationSchema, (notification) => { - log.debug('Notification received', { level: notification.params.level, data: notification.params.data }); - }); - - // List and call tools - await listTools(client); - - await callSearchTool(client); - await callActor(client); - - // Keep the connection open to receive notifications - log.debug('\nKeeping connection open to receive notifications. Press Ctrl+C to exit.'); -} - -async function listTools(client: Client): Promise { - try { - const toolsRequest: ListToolsRequest = { - method: 'tools/list', - params: {}, - }; - const toolsResult = await client.request(toolsRequest, ListToolsResultSchema); - log.debug('Tools available', { itemCount: toolsResult.tools.length }); - for (const tool of toolsResult.tools) { - log.debug('Tool detail', { toolName: tool.name, description: tool.description }); - } - if (toolsResult.tools.length === 0) { - log.debug('No tools available from the server'); - } - } catch (error) { - log.error('Tools not supported by this server', { error }); - } -} - -async function callSearchTool(client: Client): Promise { - try { - const searchRequest: CallToolRequest = { - method: 'tools/call', - params: { - name: HelperTools.STORE_SEARCH, - arguments: { search: 'rag web browser', limit: 1 }, - }, - }; - const searchResult = await client.request(searchRequest, CallToolResultSchema); - log.debug('Search result:'); - const resultContent = searchResult.content || []; - resultContent.forEach((item) => { - if (item.type === 'text') { - log.debug('Search result item', { text: item.text }); - } - }); - } catch (error) { - log.error('Error calling greet tool', { error }); - } -} - -async function callActor(client: Client): Promise { - try { - log.debug('\nCalling Actor...'); - const actorRequest: CallToolRequest = { - method: 'tools/call', - params: { - name: 'apify/rag-web-browser', - arguments: { query: 'apify mcp server' }, - }, - }; - const actorResult = await client.request(actorRequest, CallToolResultSchema); - log.debug('Actor results:'); - const resultContent = actorResult.content || []; - resultContent.forEach((item) => { - if (item.type === 'text') { - log.debug('Actor result item', { text: item.text }); - } - }); - } catch (error) { - log.error('Error calling Actor', { error }); - } -} - -main().catch((error: unknown) => { - log.error('Error running MCP client', { error: error as Error }); - process.exit(1); -}); diff --git a/src/examples/client_sse.py b/src/examples/client_sse.py deleted file mode 100644 index 9ffb9791..00000000 --- a/src/examples/client_sse.py +++ /dev/null @@ -1,53 +0,0 @@ -""" -Test Apify MCP Server using SSE client - -It is using python client as the typescript one does not support custom headers when connecting to the SSE server. - -Install python dependencies (assumes you have python installed): -> pip install requests python-dotenv mcp -""" - -import asyncio -import os -from pathlib import Path - -import requests -from dotenv import load_dotenv -from mcp.client.session import ClientSession -from mcp.client.sse import sse_client - -load_dotenv(Path(__file__).resolve().parent.parent.parent / ".env") - -MCP_SERVER_URL = "https://actors-mcp-server.apify.actor" - -HEADERS = {"Authorization": f"Bearer {os.getenv('APIFY_TOKEN')}"} - -async def run() -> None: - - print("Start MCP Server with Actors") - r = requests.get(MCP_SERVER_URL, headers=HEADERS) - print("MCP Server Response:", r.json(), end="\n\n") - - async with sse_client(url=f"{MCP_SERVER_URL}/sse", timeout=60, headers=HEADERS) as (read, write): - async with ClientSession(read, write) as session: - await session.initialize() - - tools = await session.list_tools() - print("Available Tools:", tools, end="\n\n") - for tool in tools.tools: - print(f"\n### Tool name ###: {tool.name}") - print(f"\tdescription: {tool.description}") - print(f"\tinputSchema: {tool.inputSchema}") - - if hasattr(tools, "tools") and not tools.tools: - print("No tools available!") - return - - print("\n\nCall tool") - result = await session.call_tool("apify/rag-web-browser", { "query": "example.com", "maxResults": 3 }) - print("Tools call result:") - - for content in result.content: - print(content) - -asyncio.run(run()) diff --git a/src/index-internals.ts b/src/index-internals.ts index cb9d27ea..364fe5ae 100644 --- a/src/index-internals.ts +++ b/src/index-internals.ts @@ -3,19 +3,22 @@ */ import { defaults, HelperTools } from './const.js'; -import { parseInputParamsFromUrl, processParamsGetTools } from './mcp/utils.js'; -import { addRemoveTools, defaultTools, getActorsAsTools, toolCategories, toolCategoriesEnabledByDefault } from './tools/index.js'; +import { processParamsGetTools } from './mcp/utils.js'; +import { addTool } from './tools/helpers.js'; +import { defaultTools, getActorsAsTools, toolCategories, toolCategoriesEnabledByDefault } from './tools/index.js'; import { actorNameToToolName } from './tools/utils.js'; import type { ToolCategory } from './types.js'; -import { getToolPublicFieldOnly } from './utils/tools.js'; +import { getExpectedToolNamesByCategories, getToolPublicFieldOnly } from './utils/tools.js'; +import { TTLLRUCache } from './utils/ttl-lru.js'; export { - parseInputParamsFromUrl, + getExpectedToolNamesByCategories, + TTLLRUCache, actorNameToToolName, HelperTools, defaults, defaultTools, - addRemoveTools, + addTool, toolCategories, toolCategoriesEnabledByDefault, type ToolCategory, diff --git a/src/input.ts b/src/input.ts index a6a63812..dc9a3e5f 100644 --- a/src/input.ts +++ b/src/input.ts @@ -1,37 +1,78 @@ /* * Actor input processing. + * + * Normalizes raw inputs (CLI/env/HTTP) into a consistent `Input` shape. + * No tool-loading is done here; we only canonicalize values and preserve + * intent via `undefined` (use defaults later) vs empty (explicitly none). */ import log from '@apify/log'; -import type { Input, ToolCategory } from './types.js'; +import type { Input, ToolSelector } from './types.js'; + +// Helpers +// Normalize booleans that may arrive as strings or be undefined. +export function toBoolean(value: unknown, defaultValue: boolean): boolean { + if (value === undefined) return defaultValue; + if (typeof value === 'boolean') return value; + if (typeof value === 'string') return value.toLowerCase() === 'true'; + return defaultValue; +} + +// Normalize lists from comma-separated strings or arrays. +export function normalizeList(value: string | unknown[] | undefined): string[] | undefined { + if (value === undefined) return undefined; + if (Array.isArray(value)) return value.map((s) => String(s).trim()).filter((s) => s !== ''); + const trimmed = String(value).trim(); + if (trimmed === '') return []; + return trimmed.split(',').map((s) => s.trim()).filter((s) => s !== ''); +} /** - * Process input parameters, split Actors string into an array - * @param originalInput - * @returns input + * Normalize user-provided input into a canonical `Input`. + * + * Responsibilities: + * - Coerce `actors`, `tools` from string/array into trimmed arrays ('' → []). + * - Normalize booleans (including legacy `enableActorAutoLoading`). + * - Merge `actors` into `tools` so selection lives in one place. + * + * Semantics passed to the loader: + * - `undefined` → use defaults; `[]` → explicitly none. */ export function processInput(originalInput: Partial): Input { - const input = originalInput as Input; + // Normalize actors (strings and arrays) to a clean array or undefined + const actors = normalizeList(originalInput.actors) as unknown as string[] | undefined; - // actors can be a string or an array of strings - if (input.actors && typeof input.actors === 'string') { - input.actors = input.actors.split(',').map((format: string) => format.trim()) as string[]; + // Map deprecated flag to the new one and normalize both to boolean. + let enableAddingActors: boolean; + if (originalInput.enableAddingActors === undefined && originalInput.enableActorAutoLoading !== undefined) { + log.warning('enableActorAutoLoading is deprecated, use enableAddingActors instead'); + enableAddingActors = toBoolean(originalInput.enableActorAutoLoading, false); + } else { + enableAddingActors = toBoolean(originalInput.enableAddingActors, false); } - // enableAddingActors is deprecated, use enableActorAutoLoading instead - if (input.enableAddingActors === undefined) { - if (input.enableActorAutoLoading !== undefined) { - log.warning('enableActorAutoLoading is deprecated, use enableAddingActors instead'); - input.enableAddingActors = input.enableActorAutoLoading === true || input.enableActorAutoLoading === 'true'; + // Normalize tools (strings/arrays) to a clean array or undefined + let tools = normalizeList(originalInput.tools as string | string[] | undefined) as unknown as ToolSelector[] | undefined; + + // Merge actors into tools. If tools undefined → tools = actors, then remove actors; + // otherwise append actors to tools. + // NOTE (future): Actor names contain '/', unlike internal tool names or categories. We could use that to differentiate between the two. + if (Array.isArray(actors) && actors.length > 0) { + if (tools === undefined) { + tools = [...actors] as ToolSelector[]; } else { - input.enableAddingActors = true; + const currentTools: ToolSelector[] = Array.isArray(tools) + ? tools + : [tools as ToolSelector]; + tools = [...currentTools, ...actors] as ToolSelector[]; } - } else { - input.enableAddingActors = input.enableAddingActors === true || input.enableAddingActors === 'true'; } - if (input.tools && typeof input.tools === 'string') { - input.tools = input.tools.split(',').map((tool: string) => tool.trim()) as ToolCategory[]; - } - return input; + // Return a new object with all properties explicitly defined + return { + ...originalInput, + actors: Array.isArray(actors) && actors.length > 0 && tools !== undefined ? undefined : actors, + enableAddingActors, + tools, + }; } diff --git a/src/main.ts b/src/main.ts index 5012ec60..0d620519 100644 --- a/src/main.ts +++ b/src/main.ts @@ -29,21 +29,9 @@ const input = processInput((await Actor.getInput>()) ?? ({} as In log.info('Loaded input', { input: JSON.stringify(input) }); if (STANDBY_MODE) { - let actorsToLoad: string[] = []; - // TODO: in standby mode the input loading does not actually work, - // we should remove this since we are using the URL query parameters to load Actors - // Load only Actors specified in the input - // If you wish to start without any Actor, create a task and leave the input empty - if (input.actors && input.actors.length > 0) { - const { actors } = input; - actorsToLoad = Array.isArray(actors) ? actors : actors.split(','); - } - // Include Actors to load in the MCP server options for backwards compatibility - const app = createExpressApp(HOST, { - enableAddingActors: Boolean(input.enableAddingActors), - enableDefaultActors: false, - actors: actorsToLoad, - }); + // In standby mode, actors and tools are provided via URL query params per request + // Start express app + const app = createExpressApp(HOST); log.info('Actor is running in the STANDBY mode.'); app.listen(PORT, () => { diff --git a/src/mcp/server.ts b/src/mcp/server.ts index 36c9f50c..ef642063 100644 --- a/src/mcp/server.ts +++ b/src/mcp/server.ts @@ -21,13 +21,12 @@ import { type ActorCallOptions, ApifyApiError } from 'apify-client'; import log from '@apify/log'; import { - defaults, SERVER_NAME, SERVER_VERSION, } from '../const.js'; import { prompts } from '../prompts/index.js'; -import { addRemoveTools, callActorGetDataset, defaultTools, getActorsAsTools, toolCategories } from '../tools/index.js'; -import { actorNameToToolName, decodeDotPropertyNames } from '../tools/utils.js'; +import { callActorGetDataset, defaultTools, getActorsAsTools, toolCategories } from '../tools/index.js'; +import { decodeDotPropertyNames } from '../tools/utils.js'; import type { ActorMcpTool, ActorTool, HelperTool, ToolEntry } from '../types.js'; import { createProgressTracker } from '../utils/progress.js'; import { getToolPublicFieldOnly } from '../utils/tools.js'; @@ -35,11 +34,6 @@ import { connectMCPClient } from './client.js'; import { EXTERNAL_TOOL_CALL_TIMEOUT_MSEC } from './const.js'; import { processParamsGetTools } from './utils.js'; -type ActorsMcpServerOptions = { - enableAddingActors?: boolean; - enableDefaultActors?: boolean; -}; - type ToolsChangedHandler = (toolNames: string[]) => void; /** @@ -48,15 +42,10 @@ type ToolsChangedHandler = (toolNames: string[]) => void; export class ActorsMcpServer { public readonly server: Server; public readonly tools: Map; - private options: ActorsMcpServerOptions; private toolsChangedHandler: ToolsChangedHandler | undefined; private sigintHandler: (() => Promise) | undefined; - constructor(options: ActorsMcpServerOptions = {}, setupSigintHandler = true) { - this.options = { - enableAddingActors: options.enableAddingActors ?? true, - enableDefaultActors: options.enableDefaultActors ?? true, // Default to true for backward compatibility - }; + constructor(setupSigintHandler = true) { this.server = new Server( { name: SERVER_NAME, @@ -74,19 +63,6 @@ export class ActorsMcpServer { this.setupErrorHandling(setupSigintHandler); this.setupToolHandlers(); this.setupPromptHandlers(); - - // Add default tools - this.upsertTools(defaultTools); - - // Add tools to dynamically load Actors - if (this.options.enableAddingActors) { - this.enableDynamicActorTools(); - } - - // Initialize automatically for backward compatibility - this.initialize().catch((error) => { - log.error('Failed to initialize server', { error }); - }); } /** @@ -175,7 +151,6 @@ export class ActorsMcpServer { const toolsToLoad: ToolEntry[] = []; const internalToolMap = new Map([ ...defaultTools, - ...addRemoveTools, ...Object.values(toolCategories).flat(), ].map((tool) => [tool.tool.name, tool])); @@ -195,61 +170,23 @@ export class ActorsMcpServer { } if (actorsToLoad.length > 0) { - const actorTools = await getActorsAsTools(actorsToLoad, apifyToken); - if (actorTools.length > 0) { - this.upsertTools(actorTools); - } - } - } - - /** - * Resets the server to the default state. - * This method clears all tools and loads the default tools. - * Used primarily for testing purposes. - */ - public async reset(): Promise { - this.tools.clear(); - // Unregister the tools changed handler - if (this.toolsChangedHandler) { - this.unregisterToolsChangedHandler(); - } - this.upsertTools(defaultTools); - if (this.options.enableAddingActors) { - this.enableDynamicActorTools(); + await this.loadActorsAsTools(actorsToLoad, apifyToken); } - // Initialize automatically for backward compatibility - await this.initialize(); } /** - * Initialize the server with default tools if enabled - */ - public async initialize(): Promise { - if (this.options.enableDefaultActors) { - await this.loadDefaultActors(process.env.APIFY_TOKEN as string); - } - } - - /** - * Loads default tools if not already loaded. + * Load actors as tools, upsert them to the server, and return the tool entries. + * This is a public method that wraps getActorsAsTools and handles the upsert operation. + * @param actorIdsOrNames - Array of actor IDs or names to load as tools * @param apifyToken - Apify API token for authentication - * @returns {Promise} - A promise that resolves when the tools are loaded + * @returns Promise - Array of loaded tool entries */ - public async loadDefaultActors(apifyToken: string): Promise { - const missingActors = defaults.actors.filter((name) => !this.tools.has(actorNameToToolName(name))); - const tools = await getActorsAsTools(missingActors, apifyToken); - if (tools.length > 0) { - log.debug('Loading default tools'); - this.upsertTools(tools); + public async loadActorsAsTools(actorIdsOrNames: string[], apifyToken: string): Promise { + const actorTools = await getActorsAsTools(actorIdsOrNames, apifyToken); + if (actorTools.length > 0) { + this.upsertTools(actorTools, true); } - } - - /** - * @deprecated Use `loadDefaultActors` instead. - * Loads default tools if not already loaded. - */ - public async loadDefaultTools(apifyToken: string) { - await this.loadDefaultActors(apifyToken); + return actorTools; } /** @@ -267,19 +204,6 @@ export class ActorsMcpServer { } } - /** - * Add Actors to server dynamically - */ - public enableDynamicActorTools() { - this.options.enableAddingActors = true; - this.upsertTools(addRemoveTools, false); - } - - public disableDynamicActorTools() { - this.options.enableAddingActors = false; - this.removeToolsByName(addRemoveTools.map((tool) => tool.tool.name)); - } - /** Delete tools from the server and notify the handler. */ public removeToolsByName(toolNames: string[], shouldNotifyToolsChangedHandler = false): string[] { diff --git a/src/stdio.ts b/src/stdio.ts index a4a52c02..39bb309a 100644 --- a/src/stdio.ts +++ b/src/stdio.ts @@ -22,9 +22,9 @@ import { hideBin } from 'yargs/helpers'; import log from '@apify/log'; +import { processInput } from './input.js'; import { ActorsMcpServer } from './mcp/server.js'; -import { toolCategories } from './tools/index.js'; -import type { Input, ToolCategory } from './types.js'; +import type { Input, ToolSelector } from './types.js'; import { loadToolsFromInput } from './utils/tools-loader.js'; // Keeping this interface here and not types.ts since @@ -46,6 +46,7 @@ log.setLevel(log.LEVELS.ERROR); // Parse command line arguments using yargs const argv = yargs(hideBin(process.argv)) + .wrap(null) // Disable automatic wrapping to avoid issues with long lines and links .usage('Usage: $0 [options]') .env() .option('actors', { @@ -55,30 +56,22 @@ const argv = yargs(hideBin(process.argv)) }) .option('enable-adding-actors', { type: 'boolean', - default: true, - describe: 'Enable dynamically adding Actors as tools based on user requests. Can also be set via ENABLE_ADDING_ACTORS environment variable.', + default: false, + describe: `Enable dynamically adding Actors as tools based on user requests. Can also be set via ENABLE_ADDING_ACTORS environment variable. +Deprecated: use tools experimental category instead.`, }) .option('enableActorAutoLoading', { type: 'boolean', - default: true, + default: false, hidden: true, describe: 'Deprecated: use enable-adding-actors instead.', }) .options('tools', { type: 'string', - describe: `Comma-separated list of specific tool categories to enable. Can also be set via TOOLS environment variable. + describe: `Comma-separated list of tools to enable. Can be either a tool category, a specific tool, or an Apify Actor. For example: --tools actors,docs,apify/rag-web-browser. Can also be set via TOOLS environment variable. -Available choices: ${Object.keys(toolCategories).join(', ')} - -Tool categories are as follows: -- docs: Search and fetch Apify documentation tools. -- runs: Get Actor runs list, run details, and logs from a specific Actor run. -- storage: Access datasets, key-value stores, and their records. -- preview: Experimental tools in preview mode. - -Note: Tools that enable you to search Actors from the Apify Store and get their details are always enabled by default. -`, - example: 'docs,runs,storage', +For more details visit https://mcp.apify.com`, + example: 'actors,docs,apify/rag-web-browser', }) .help('help') .alias('h', 'help') @@ -90,11 +83,16 @@ Note: Tools that enable you to search Actors from the Apify Store and get their .epilogue('For more information, visit https://mcp.apify.com or https://github.com/apify/apify-mcp-server') .parseSync() as CliArgs; -const enableAddingActors = argv.enableAddingActors && argv.enableActorAutoLoading; -const actors = argv.actors as string || ''; -const actorList = actors ? actors.split(',').map((a: string) => a.trim()) : []; -// Keys of the tool categories to enable -const toolCategoryKeys = argv.tools ? argv.tools.split(',').map((t: string) => t.trim()) : []; +// Respect either the new flag or the deprecated one +const enableAddingActors = Boolean(argv.enableAddingActors || argv.enableActorAutoLoading); +// Split actors argument, trim whitespace, and filter out empty strings +const actorList = argv.actors !== undefined + ? argv.actors.split(',').map((a: string) => a.trim()).filter((a: string) => a.length > 0) + : undefined; +// Split tools argument, trim whitespace, and filter out empty strings +const toolCategoryKeys = argv.tools !== undefined + ? argv.tools.split(',').map((t: string) => t.trim()).filter((t: string) => t.length > 0) + : undefined; // Propagate log.error to console.error for easier debugging const originalError = log.error.bind(log); @@ -111,17 +109,20 @@ if (!process.env.APIFY_TOKEN) { } async function main() { - const mcpServer = new ActorsMcpServer({ enableAddingActors, enableDefaultActors: false }); + const mcpServer = new ActorsMcpServer(); // Create an Input object from CLI arguments const input: Input = { - actors: actorList.length ? actorList : [], + actors: actorList, enableAddingActors, - tools: toolCategoryKeys as ToolCategory[], + tools: toolCategoryKeys as ToolSelector[], }; + // Normalize (merges actors into tools for backward compatibility) + const normalized = processInput(input); + // Use the shared tools loading logic - const tools = await loadToolsFromInput(input, process.env.APIFY_TOKEN as string, actorList.length === 0); + const tools = await loadToolsFromInput(normalized, process.env.APIFY_TOKEN as string); mcpServer.upsertTools(tools); diff --git a/src/tools/actor.ts b/src/tools/actor.ts index f14f2726..c3f1a7e4 100644 --- a/src/tools/actor.ts +++ b/src/tools/actor.ts @@ -18,15 +18,11 @@ import { getMCPServerTools } from '../mcp/proxy.js'; import { actorDefinitionPrunedCache } from '../state.js'; import type { ActorDefinitionStorage, ActorInfo, ToolEntry } from '../types.js'; import { getActorDefinitionStorageFieldNames } from '../utils/actor.js'; +import { fetchActorDetails } from '../utils/actor-details.js'; import { getValuesByDotKeys } from '../utils/generic.js'; import type { ProgressTracker } from '../utils/progress.js'; import { getActorDefinition } from './build.js'; -import { - actorNameToToolName, - fixedAjvCompile, - getToolSchemaID, - transformActorInputSchemaProperties, -} from './utils.js'; +import { actorNameToToolName, fixedAjvCompile, getToolSchemaID, transformActorInputSchemaProperties } from './utils.js'; const ajv = new Ajv({ coerceTypes: 'array', strict: false }); @@ -141,7 +137,9 @@ export async function getNormalActorsAsTools( tool: { name: actorNameToToolName(actorDefinitionPruned.actorFullName), actorFullName: actorDefinitionPruned.actorFullName, - description: `${actorDefinitionPruned.description} Instructions: ${ACTOR_ADDITIONAL_INSTRUCTIONS}`, + description: `This tool calls the Actor "${actorDefinitionPruned.actorFullName}" and retrieves its output results. Use this tool instead of the "${HelperTools.ACTOR_CALL}" if user requests to use this specific Actor. +Actor description: ${actorDefinitionPruned.description} +Instructions: ${ACTOR_ADDITIONAL_INSTRUCTIONS}`, inputSchema: actorDefinitionPruned.input // So Actor without input schema works - MCP client expects JSON schema valid output || { @@ -246,14 +244,25 @@ export async function getActorsAsTools( const callActorArgs = z.object({ actor: z.string() - .describe('The name of the Actor to call. For example, "apify/instagram-scraper".'), + .describe('The name of the Actor to call. For example, "apify/rag-web-browser".'), + step: z.enum(['info', 'call']) + .default('info') + .describe(`Step to perform: "info" to get Actor details and input schema (required first step), "call" to execute the Actor (only after getting info).`), input: z.object({}).passthrough() - .describe('The input JSON to pass to the Actor. For example, {"query": "apify", "maxItems": 10}.'), + .optional() + .describe(`The input JSON to pass to the Actor. For example, {"query": "apify", "maxResults": 5, "outputFormats": ["markdown"]}. Required only when step is "call".`), callOptions: z.object({ - memory: z.number().optional(), - timeout: z.number().optional(), + memory: z.number() + .min(128, 'Memory must be at least 128 MB') + .max(32768, 'Memory cannot exceed 32 GB (32768 MB)') + .optional() + .describe(`Memory allocation for the Actor in MB. Must be a power of 2 (e.g., 128, 256, 512, 1024, 2048, 4096, 8192, 16384, 32768). Minimum: 128 MB, Maximum: 32768 MB (32 GB).`), + timeout: z.number() + .min(0, 'Timeout must be 0 or greater') + .optional() + .describe(`Maximum runtime for the Actor in seconds. After this time elapses, the Actor will be automatically terminated. Use 0 for infinite timeout (no time limit). Minimum: 0 seconds (infinite).`), }).optional() - .describe('Optional call options for the Actor.'), + .describe('Optional call options for the Actor run configuration.'), }); export const callActor: ToolEntry = { @@ -261,38 +270,56 @@ export const callActor: ToolEntry = { tool: { name: HelperTools.ACTOR_CALL, actorFullName: HelperTools.ACTOR_CALL, - description: `Call an Actor and get the Actor run results. If you are not sure about the Actor input, you MUST get the Actor details first, which also returns the input schema using ${HelperTools.ACTOR_GET_DETAILS}. The Actor MUST be added before calling; use the ${HelperTools.ACTOR_ADD} tool first. By default, the Apify MCP server makes newly added Actors available as tools for calling. Use this tool ONLY if you cannot call the newly added tool directly, and NEVER call this tool before first trying to call the tool directly. For example, when you add an Actor "apify/instagram-scraper" using the ${HelperTools.ACTOR_ADD} tool, the Apify MCP server will add a new tool ${actorNameToToolName('apify/instagram-scraper')} that you can call directly. If calling this tool does not work, then and ONLY then MAY you use this tool as a backup.`, + description: `Call Any Actor from Apify Store - Two-Step Process + +This tool uses a mandatory two-step process to safely call any Actor from the Apify store. + +USAGE: +• ONLY for Actors that are NOT available as dedicated tools +• If a dedicated tool exists (e.g., ${actorNameToToolName('apify/rag-web-browser')}), use that instead + +MANDATORY TWO-STEP WORKFLOW: + +Step 1: Get Actor Info (step="info", default) +• First call this tool with step="info" to get Actor details and input schema +• This returns the Actor description, documentation, and required input schema +• You MUST do this step first - it's required to understand how to call the Actor + +Step 2: Call Actor (step="call") +• Only after step 1, call again with step="call" and proper input based on the schema +• This executes the Actor and returns the results + +The step parameter enforces this workflow - you cannot call an Actor without first getting its info.`, inputSchema: zodToJsonSchema(callActorArgs), ajvValidate: ajv.compile(zodToJsonSchema(callActorArgs)), call: async (toolArgs) => { - const { apifyMcpServer, args, apifyToken, progressTracker } = toolArgs; - const { actor: actorName, input, callOptions } = callActorArgs.parse(args); + const { args, apifyToken, progressTracker } = toolArgs; + const { actor: actorName, step, input, callOptions } = callActorArgs.parse(args); - const actors = apifyMcpServer.listActorToolNames(); - if (!actors.includes(actorName)) { - const toolsText = actors.length > 0 ? `Available Actors are: ${actors.join(', ')}` : 'No Actors have been added yet.'; - if (apifyMcpServer.tools.has(HelperTools.ACTOR_ADD)) { + try { + if (step === 'info') { + // Step 1: Return actor card and schema directly + const details = await fetchActorDetails(apifyToken, actorName); + if (!details) { + return { + content: [{ type: 'text', text: `Actor information for '${actorName}' was not found. Please check the Actor ID or name and ensure the Actor exists.` }], + }; + } + return { + content: [ + { type: 'text', text: `**Input Schema:**\n${JSON.stringify(details.inputSchema, null, 0)}` }, + ], + }; + } + // Step 2: Call the Actor + if (!input) { return { - content: [{ - type: 'text', - text: `Actor '${actorName}' is not added. Add it with the '${HelperTools.ACTOR_ADD}' tool. ${toolsText}`, - }], + content: [ + { type: 'text', text: `Input is required when step="call". Please provide the input parameter based on the Actor's input schema.` }, + ], }; } - return { - content: [{ - type: 'text', - text: `Actor '${actorName}' is not added. ${toolsText}\n` - + 'To use this MCP server, specify the actors with the parameter, for example:\n' - + '?actors=apify/instagram-scraper,apify/website-content-crawler\n' - + 'or with the CLI:\n' - + '--actors "apify/instagram-scraper,apify/website-content-crawler"\n' - + 'You can only use actors that are included in the list; actors not in the list cannot be used.', - }], - }; - } - try { const [actor] = await getActorsAsTools([actorName], apifyToken); if (!actor) { @@ -315,7 +342,7 @@ export const callActor: ToolEntry = { } } - const { items } = await callActorGetDataset( + const { runId, datasetId, items } = await callActorGetDataset( actorName, input, apifyToken, @@ -323,17 +350,22 @@ export const callActor: ToolEntry = { progressTracker, ); - return { - content: items.items.map((item: Record) => ({ - type: 'text', - text: JSON.stringify(item), - })), - }; + const content = [ + { type: 'text', text: `Actor finished with runId: ${runId}, datasetId ${datasetId}` }, + ]; + + const itemContents = items.items.map((item: Record) => ({ + type: 'text', + text: JSON.stringify(item), + })); + content.push(...itemContents); + + return { content }; } catch (error) { - log.error('Error calling Actor', { error }); + log.error('Error with Actor operation', { error, actorName, step }); return { content: [ - { type: 'text', text: `Error calling Actor: ${error instanceof Error ? error.message : String(error)}` }, + { type: 'text', text: `Error with Actor operation: ${error instanceof Error ? error.message : String(error)}` }, ], }; } diff --git a/src/tools/fetch-actor-details.ts b/src/tools/fetch-actor-details.ts new file mode 100644 index 00000000..ef3dbd74 --- /dev/null +++ b/src/tools/fetch-actor-details.ts @@ -0,0 +1,48 @@ +import { z } from 'zod'; +import zodToJsonSchema from 'zod-to-json-schema'; + +import { HelperTools } from '../const.js'; +import type { InternalTool, ToolEntry } from '../types.js'; +import { fetchActorDetails } from '../utils/actor-details.js'; +import { ajv } from '../utils/ajv.js'; + +const fetchActorDetailsToolArgsSchema = z.object({ + actor: z.string() + .min(1) + .describe(`Actor ID or full name in the format "username/name", e.g., "apify/rag-web-browser".`), +}); + +export const fetchActorDetailsTool: ToolEntry = { + type: 'internal', + tool: { + name: HelperTools.ACTOR_GET_DETAILS, + description: `Get detailed information about an Actor by its ID or full name.\n` + + `This tool returns title, description, URL, README (Actor's documentation), input schema, and usage statistics. \n` + + `The Actor name is always composed of "username/name", for example, "apify/rag-web-browser".\n` + + `Present Actor information in user-friendly format as an Actor card.\n` + + `USAGE:\n` + + `- Use when user asks about an Actor its details, description, input schema, etc.\n` + + `EXAMPLES:\n` + + `- user_input: How to use apify/rag-web-browser\n` + + `- user_input: What is the input schema for apify/rag-web-browser`, + inputSchema: zodToJsonSchema(fetchActorDetailsToolArgsSchema), + ajvValidate: ajv.compile(zodToJsonSchema(fetchActorDetailsToolArgsSchema)), + call: async (toolArgs) => { + const { args, apifyToken } = toolArgs; + const parsed = fetchActorDetailsToolArgsSchema.parse(args); + const details = await fetchActorDetails(apifyToken, parsed.actor); + if (!details) { + return { + content: [{ type: 'text', text: `Actor information for '${parsed.actor}' was not found. Please check the Actor ID or name and ensure the Actor exists.` }], + }; + } + return { + content: [ + { type: 'text', text: `**Actor card**:\n${details.actorCard}` }, + { type: 'text', text: `**README:**\n${details.readme}` }, + { type: 'text', text: `**Input Schema:**\n${JSON.stringify(details.inputSchema, null, 0)}` }, + ], + }; + }, + } as InternalTool, +}; diff --git a/src/tools/get-actor-details.ts b/src/tools/get-actor-details.ts deleted file mode 100644 index 6b74d298..00000000 --- a/src/tools/get-actor-details.ts +++ /dev/null @@ -1,69 +0,0 @@ -import type { Actor, Build } from 'apify-client'; -import { z } from 'zod'; -import zodToJsonSchema from 'zod-to-json-schema'; - -import { ApifyClient } from '../apify-client.js'; -import { HelperTools } from '../const.js'; -import type { IActorInputSchema, InternalTool, ToolEntry } from '../types.js'; -import { formatActorToActorCard } from '../utils/actor-card.js'; -import { ajv } from '../utils/ajv.js'; -import { filterSchemaProperties, shortenProperties } from './utils.js'; - -const getActorDetailsToolArgsSchema = z.object({ - actor: z.string() - .min(1) - .describe(`Actor ID or full name in the format "username/name", e.g., "apify/rag-web-browser".`), -}); - -export const getActorDetailsTool: ToolEntry = { - type: 'internal', - tool: { - name: HelperTools.ACTOR_GET_DETAILS, - description: `Get detailed information about an Actor by its ID or full name.\n` - + `This tool returns title, description, URL, README (Actor's documentation), input schema, and usage statistics. \n` - + `The Actor name is always composed of "username/name", for example, "apify/rag-web-browser".\n` - + `Present Actor information in user-friendly format as an Actor card.\n` - + `USAGE:\n` - + `- Use when user asks about an Actor its details, description, input schema, etc.\n` - + `EXAMPLES:\n` - + `- user_input: How to use apify/rag-web-browser\n` - + `- user_input: What is the input schema for apify/rag-web-browser`, - inputSchema: zodToJsonSchema(getActorDetailsToolArgsSchema), - ajvValidate: ajv.compile(zodToJsonSchema(getActorDetailsToolArgsSchema)), - call: async (toolArgs) => { - const { args, apifyToken } = toolArgs; - - const parsed = getActorDetailsToolArgsSchema.parse(args); - const client = new ApifyClient({ token: apifyToken }); - - const [actorInfo, buildInfo]: [Actor | undefined, Build | undefined] = await Promise.all([ - client.actor(parsed.actor).get(), - client.actor(parsed.actor).defaultBuild().then(async (build) => build.get()), - ]); - - if (!actorInfo || !buildInfo || !buildInfo.actorDefinition) { - return { - content: [{ type: 'text', text: `Actor information for '${parsed.actor}' was not found. Please check the Actor ID or name and ensure the Actor exists.` }], - }; - } - - const inputSchema = (buildInfo.actorDefinition.input || { - type: 'object', - properties: {}, - }) as IActorInputSchema; - inputSchema.properties = filterSchemaProperties(inputSchema.properties); - inputSchema.properties = shortenProperties(inputSchema.properties); - - // Use the actor formatter to get the main actor details - const actorCard = formatActorToActorCard(actorInfo); - - return { - content: [ - { type: 'text', text: `**Actor card**:\n${actorCard}` }, - { type: 'text', text: `**README:**\n${buildInfo.actorDefinition.readme || 'No README provided.'}` }, - { type: 'text', text: `**Input Schema:**\n${JSON.stringify(inputSchema, null, 0)}` }, - ], - }; - }, - } as InternalTool, -}; diff --git a/src/tools/helpers.ts b/src/tools/helpers.ts index ccba3cdf..cd77502e 100644 --- a/src/tools/helpers.ts +++ b/src/tools/helpers.ts @@ -4,7 +4,6 @@ import zodToJsonSchema from 'zod-to-json-schema'; import { HelperTools } from '../const.js'; import type { InternalTool, ToolEntry } from '../types'; -import { getActorsAsTools } from './actor.js'; import { actorNameToToolName } from './utils.js'; const ajv = new Ajv({ coerceTypes: 'array', strict: false }); @@ -59,11 +58,12 @@ export const addTool: ToolEntry = { }], }; } - const tools = await getActorsAsTools([parsed.actor], apifyToken); + + const tools = await apifyMcpServer.loadActorsAsTools([parsed.actor], apifyToken); /** * If no tools were found, return a message that the Actor was not found * instead of returning that non existent tool was added since the - * getActorsAsTools function returns an empty array and does not throw an error. + * loadActorsAsTools method returns an empty array and does not throw an error. */ if (tools.length === 0) { return { @@ -73,14 +73,14 @@ export const addTool: ToolEntry = { }], }; } - const toolsAdded = apifyMcpServer.upsertTools(tools, true); + await sendNotification({ method: 'notifications/tools/list_changed' }); return { content: [{ type: 'text', text: `Actor ${parsed.actor} has been added. Newly available tools: ${ - toolsAdded.map( + tools.map( (t) => `${t.tool.name}`, ).join(', ') }.`, diff --git a/src/tools/index.ts b/src/tools/index.ts index 3085b860..709968bf 100644 --- a/src/tools/index.ts +++ b/src/tools/index.ts @@ -1,10 +1,11 @@ // Import specific tools that are being used import type { ToolCategory } from '../types.js'; +import { getExpectedToolsByCategories } from '../utils/tools.js'; import { callActor, callActorGetDataset, getActorsAsTools } from './actor.js'; import { getDataset, getDatasetItems, getDatasetSchema } from './dataset.js'; import { getUserDatasetsList } from './dataset_collection.js'; +import { fetchActorDetailsTool } from './fetch-actor-details.js'; import { fetchApifyDocsTool } from './fetch-apify-docs.js'; -import { getActorDetailsTool } from './get-actor-details.js'; import { addTool } from './helpers.js'; import { getKeyValueStore, getKeyValueStoreKeys, getKeyValueStoreRecord } from './key_value_store.js'; import { getUserKeyValueStoresList } from './key_value_store_collection.js'; @@ -14,6 +15,14 @@ import { searchApifyDocsTool } from './search-apify-docs.js'; import { searchActors } from './store_collection.js'; export const toolCategories = { + experimental: [ + addTool, + ], + actors: [ + fetchActorDetailsTool, + searchActors, + callActor, + ], docs: [ searchApifyDocsTool, fetchApifyDocsTool, @@ -33,24 +42,13 @@ export const toolCategories = { getUserDatasetsList, getUserKeyValueStoresList, ], - preview: [ - callActor, - ], }; export const toolCategoriesEnabledByDefault: ToolCategory[] = [ + 'actors', 'docs', ]; -export const defaultTools = [ - getActorDetailsTool, - searchActors, - // Add the tools from the enabled categories - ...toolCategoriesEnabledByDefault.map((key) => toolCategories[key]).flat(), -]; - -export const addRemoveTools = [ - addTool, -]; +export const defaultTools = getExpectedToolsByCategories(toolCategoriesEnabledByDefault); // Export only the tools that are being used export { diff --git a/src/types.ts b/src/types.ts index 497cd515..c4c5e2e8 100644 --- a/src/types.ts +++ b/src/types.ts @@ -211,9 +211,18 @@ export interface InternalTool extends ToolBase { } export type ToolCategory = keyof typeof toolCategories; +/** + * Selector for tools input - can be a category key or a specific tool name. + */ +export type ToolSelector = ToolCategory | string; export type Input = { - actors: string[] | string; + /** + * When `actors` is undefined that means the default Actors should be loaded. + * If it as empty string or empty array then no Actors should be loaded. + * Otherwise the specified Actors should be loaded. + */ + actors?: string[] | string; /** * @deprecated Use `enableAddingActors` instead. */ @@ -222,8 +231,13 @@ export type Input = { maxActorMemoryBytes?: number; debugActor?: string; debugActorInput?: unknown; - /** Tool categories to include */ - tools?: ToolCategory[] | string; + /** + * Tool selectors to include (category keys or concrete tool names). + * When `tools` is undefined that means the default tool categories should be loaded. + * If it is an empty string or empty array then no internal tools should be loaded. + * Otherwise the specified categories and/or concrete tool names should be loaded. + */ + tools?: ToolSelector[] | string; }; // Utility type to get a union of values from an object type diff --git a/src/utils/actor-details.ts b/src/utils/actor-details.ts new file mode 100644 index 00000000..494db6fb --- /dev/null +++ b/src/utils/actor-details.ts @@ -0,0 +1,38 @@ +import type { Actor, Build } from 'apify-client'; + +import { ApifyClient } from '../apify-client.js'; +import { filterSchemaProperties, shortenProperties } from '../tools/utils.js'; +import type { IActorInputSchema } from '../types.js'; +import { formatActorToActorCard } from './actor-card.js'; + +// Keep the interface here since it is a self contained module +export interface ActorDetailsResult { + actorInfo: Actor; + buildInfo: Build; + actorCard: string; + inputSchema: IActorInputSchema; + readme: string; +} + +export async function fetchActorDetails(apifyToken: string, actorName: string): Promise { + const client = new ApifyClient({ token: apifyToken }); + const [actorInfo, buildInfo]: [Actor | undefined, Build | undefined] = await Promise.all([ + client.actor(actorName).get(), + client.actor(actorName).defaultBuild().then(async (build) => build.get()), + ]); + if (!actorInfo || !buildInfo || !buildInfo.actorDefinition) return null; + const inputSchema = (buildInfo.actorDefinition.input || { + type: 'object', + properties: {}, + }) as IActorInputSchema; + inputSchema.properties = filterSchemaProperties(inputSchema.properties); + inputSchema.properties = shortenProperties(inputSchema.properties); + const actorCard = formatActorToActorCard(actorInfo); + return { + actorInfo, + buildInfo, + actorCard, + inputSchema, + readme: buildInfo.actorDefinition.readme || 'No README provided.', + }; +} diff --git a/src/utils/tools-loader.ts b/src/utils/tools-loader.ts index 897a206e..ab9c99cb 100644 --- a/src/utils/tools-loader.ts +++ b/src/utils/tools-loader.ts @@ -4,8 +4,22 @@ */ import { defaults } from '../const.js'; -import { addRemoveTools, getActorsAsTools, toolCategories } from '../tools/index.js'; +import { addTool } from '../tools/helpers.js'; +import { getActorsAsTools, toolCategories, toolCategoriesEnabledByDefault } from '../tools/index.js'; import type { Input, ToolCategory, ToolEntry } from '../types.js'; +import { getExpectedToolsByCategories } from './tools.js'; + +// Lazily-computed cache of internal tools by name to avoid circular init issues. +let INTERNAL_TOOL_BY_NAME_CACHE: Map | null = null; +function getInternalToolByNameMap(): Map { + if (!INTERNAL_TOOL_BY_NAME_CACHE) { + const allInternal = getExpectedToolsByCategories(Object.keys(toolCategories) as ToolCategory[]); + INTERNAL_TOOL_BY_NAME_CACHE = new Map( + allInternal.map((entry) => [entry.tool.name, entry]), + ); + } + return INTERNAL_TOOL_BY_NAME_CACHE; +} /** * Load tools based on the provided Input object. @@ -13,38 +27,97 @@ import type { Input, ToolCategory, ToolEntry } from '../types.js'; * * @param input The processed Input object * @param apifyToken The Apify API token - * @param useDefaultActors Whether to use default actors if no actors are specified * @returns An array of tool entries */ export async function loadToolsFromInput( input: Input, apifyToken: string, - useDefaultActors = false, ): Promise { - let tools: ToolEntry[] = []; - - // Load actors as tools - if (input.actors && (Array.isArray(input.actors) ? input.actors.length > 0 : input.actors)) { - const actors = Array.isArray(input.actors) ? input.actors : [input.actors]; - tools = await getActorsAsTools(actors, apifyToken); - } else if (useDefaultActors) { - // Use default actors if no actors are specified and useDefaultActors is true - tools = await getActorsAsTools(defaults.actors, apifyToken); + // Helpers for readability + const normalizeSelectors = (value: Input['tools']): (string | ToolCategory)[] | undefined => { + if (value === undefined) return undefined; + return (Array.isArray(value) ? value : [value]).map(String).map((s) => s.trim()).filter((s) => s !== ''); + }; + + const selectors = normalizeSelectors(input.tools); + const selectorsProvided = selectors !== undefined; + const selectorsExplicitEmpty = selectorsProvided && (selectors as string[]).length === 0; + const addActorEnabled = input.enableAddingActors === true; + const actorsExplicitlyEmpty = (Array.isArray(input.actors) && input.actors.length === 0) || input.actors === ''; + + // Partition selectors into internal picks (by category or by name) and actor names + const internalSelections: ToolEntry[] = []; + const actorSelectorsFromTools: string[] = []; + if (selectorsProvided && !selectorsExplicitEmpty) { + for (const selector of selectors as (string | ToolCategory)[]) { + const categoryTools = toolCategories[selector as ToolCategory]; + if (categoryTools) { + internalSelections.push(...categoryTools); + continue; + } + const internalByName = getInternalToolByNameMap().get(String(selector)); + if (internalByName) { + internalSelections.push(internalByName); + continue; + } + // Treat unknown selectors as Actor IDs/full names. + // Potential heuristic (future): if (String(selector).includes('/')) => definitely an Actor. + actorSelectorsFromTools.push(String(selector)); + } } - // Add tools for adding/removing actors if enabled - if (input.enableAddingActors) { - tools.push(...addRemoveTools); + // Decide which Actors to load + let actorsFromField: string[] | undefined; + if (input.actors === undefined) { + actorsFromField = undefined; + } else if (Array.isArray(input.actors)) { + actorsFromField = input.actors; + } else { + actorsFromField = [input.actors]; } - // Add tools from enabled categories - if (input.tools) { - const toolKeys = Array.isArray(input.tools) ? input.tools : [input.tools]; - for (const toolKey of toolKeys) { - const keyTools = toolCategories[toolKey as ToolCategory] || []; - tools.push(...keyTools); + let actorNamesToLoad: string[] = []; + if (actorsFromField !== undefined) { + actorNamesToLoad = actorsFromField; + } else if (actorSelectorsFromTools.length > 0) { + actorNamesToLoad = actorSelectorsFromTools; + } else if (!selectorsProvided) { + // No selectors supplied: use defaults unless add-actor mode is enabled + actorNamesToLoad = addActorEnabled ? [] : defaults.actors; + } // else: selectors provided but none are actors => do not load defaults + + // Compose final tool list + const result: ToolEntry[] = []; + + // Internal tools + if (selectorsProvided) { + result.push(...internalSelections); + // If add-actor mode is enabled, ensure add-actor tool is available alongside selected tools. + if (addActorEnabled && !selectorsExplicitEmpty && !actorsExplicitlyEmpty) { + const hasAddActor = result.some((e) => e.tool.name === addTool.tool.name); + if (!hasAddActor) result.push(addTool); } + } else if (addActorEnabled && !actorsExplicitlyEmpty) { + // No selectors: either expose only add-actor (when enabled), or default categories + result.push(addTool); + } else if (!actorsExplicitlyEmpty) { + result.push(...getExpectedToolsByCategories(toolCategoriesEnabledByDefault)); } - return tools; + // Actor tools (if any) + if (actorNamesToLoad.length > 0) { + const actorTools = await getActorsAsTools(actorNamesToLoad, apifyToken); + result.push(...actorTools); + } + + // De-duplicate by tool name for safety + const seen = new Set(); + const deduped = result.filter((entry) => { + const { name } = entry.tool; + if (seen.has(name)) return false; + seen.add(name); + return true; + }); + + return deduped; } diff --git a/src/utils/tools.ts b/src/utils/tools.ts index 67f4b1ee..7d8f0dbf 100644 --- a/src/utils/tools.ts +++ b/src/utils/tools.ts @@ -1,4 +1,5 @@ -import type { ToolBase } from '../types.js'; +import { toolCategories } from '../tools/index.js'; +import type { ToolBase, ToolCategory, ToolEntry } from '../types.js'; /** * Returns a public version of the tool containing only fields that should be exposed publicly. @@ -11,3 +12,18 @@ export function getToolPublicFieldOnly(tool: ToolBase) { inputSchema: tool.inputSchema, }; } + +/** + * Returns the tool objects for the given category names using toolCategories. + */ +export function getExpectedToolsByCategories(categories: ToolCategory[]): ToolEntry[] { + return categories + .flatMap((category) => toolCategories[category] || []); +} + +/** + * Returns the tool names for the given category names using getExpectedToolsByCategories. + */ +export function getExpectedToolNamesByCategories(categories: ToolCategory[]): string[] { + return getExpectedToolsByCategories(categories).map((tool) => tool.tool.name); +} diff --git a/tests/const.ts b/tests/const.ts index 785c35e2..9201564c 100644 --- a/tests/const.ts +++ b/tests/const.ts @@ -1,8 +1,9 @@ import { defaults } from '../src/const.js'; -import { defaultTools } from '../src/tools/index.js'; +import { toolCategoriesEnabledByDefault } from '../src/tools/index.js'; import { actorNameToToolName } from '../src/tools/utils.js'; +import { getExpectedToolNamesByCategories } from '../src/utils/tools.js'; export const ACTOR_PYTHON_EXAMPLE = 'apify/python-example'; export const ACTOR_MCP_SERVER_ACTOR_NAME = 'apify/actors-mcp-server'; -export const DEFAULT_TOOL_NAMES = defaultTools.map((tool) => tool.tool.name); +export const DEFAULT_TOOL_NAMES = getExpectedToolNamesByCategories(toolCategoriesEnabledByDefault); export const DEFAULT_ACTOR_NAMES = defaults.actors.map((tool) => actorNameToToolName(tool)); diff --git a/tests/helpers.ts b/tests/helpers.ts index 77223da3..fe11b796 100644 --- a/tests/helpers.ts +++ b/tests/helpers.ts @@ -10,7 +10,7 @@ import type { ToolCategory } from '../src/types.js'; export interface McpClientOptions { actors?: string[]; enableAddingActors?: boolean; - tools?: ToolCategory[]; // Tool categories to include + tools?: (ToolCategory | string)[]; // Tool categories, specific tool or Actor names to include useEnv?: boolean; // Use environment variables instead of command line arguments (stdio only) } @@ -23,13 +23,13 @@ export async function createMcpSseClient( } const url = new URL(serverUrl); const { actors, enableAddingActors, tools } = options || {}; - if (actors) { + if (actors !== undefined) { url.searchParams.append('actors', actors.join(',')); } if (enableAddingActors !== undefined) { url.searchParams.append('enableAddingActors', enableAddingActors.toString()); } - if (tools && tools.length > 0) { + if (tools !== undefined) { url.searchParams.append('tools', tools.join(',')); } @@ -62,13 +62,13 @@ export async function createMcpStreamableClient( } const url = new URL(serverUrl); const { actors, enableAddingActors, tools } = options || {}; - if (actors) { + if (actors !== undefined) { url.searchParams.append('actors', actors.join(',')); } if (enableAddingActors !== undefined) { url.searchParams.append('enableAddingActors', enableAddingActors.toString()); } - if (tools && tools.length > 0) { + if (tools !== undefined) { url.searchParams.append('tools', tools.join(',')); } @@ -106,24 +106,24 @@ export async function createMcpStdioClient( // Set environment variables instead of command line arguments when useEnv is true if (useEnv) { - if (actors) { + if (actors !== undefined) { env.ACTORS = actors.join(','); } if (enableAddingActors !== undefined) { env.ENABLE_ADDING_ACTORS = enableAddingActors.toString(); } - if (tools && tools.length > 0) { + if (tools !== undefined) { env.TOOLS = tools.join(','); } } else { // Use command line arguments as before - if (actors) { + if (actors !== undefined) { args.push('--actors', actors.join(',')); } if (enableAddingActors !== undefined) { args.push('--enable-adding-actors', enableAddingActors.toString()); } - if (tools && tools.length > 0) { + if (tools !== undefined) { args.push('--tools', tools.join(',')); } } diff --git a/tests/integration/actor.server-sse.test.ts b/tests/integration/actor.server-sse.test.ts index eb783954..6142cfc1 100644 --- a/tests/integration/actor.server-sse.test.ts +++ b/tests/integration/actor.server-sse.test.ts @@ -21,12 +21,8 @@ createIntegrationTestsSuite({ beforeAllFn: async () => { log.setLevel(log.LEVELS.OFF); - // Create an express app using the proper server setup - const mcpServerOptions = { - enableAddingActors: false, - enableDefaultActors: false, - }; - app = createExpressApp(httpServerHost, mcpServerOptions); + // Create an express app + app = createExpressApp(httpServerHost); // Start a test server await new Promise((resolve) => { diff --git a/tests/integration/actor.server-streamable.test.ts b/tests/integration/actor.server-streamable.test.ts index bb1f5249..56aa5226 100644 --- a/tests/integration/actor.server-streamable.test.ts +++ b/tests/integration/actor.server-streamable.test.ts @@ -21,12 +21,8 @@ createIntegrationTestsSuite({ beforeAllFn: async () => { log.setLevel(log.LEVELS.OFF); - // Create an express app using the proper server setup - const mcpServerOptions = { - enableAddingActors: false, - enableDefaultActors: false, - }; - app = createExpressApp(httpServerHost, mcpServerOptions); + // Create an express app + app = createExpressApp(httpServerHost); // Start a test server await new Promise((resolve) => { diff --git a/tests/integration/internals.test.ts b/tests/integration/internals.test.ts index 75ff56fa..98500314 100644 --- a/tests/integration/internals.test.ts +++ b/tests/integration/internals.test.ts @@ -3,10 +3,12 @@ import { beforeAll, describe, expect, it } from 'vitest'; import log from '@apify/log'; import { actorNameToToolName } from '../../dist/tools/utils.js'; -import { defaults } from '../../src/const.js'; import { ActorsMcpServer } from '../../src/index.js'; -import { addRemoveTools, defaultTools, getActorsAsTools } from '../../src/tools/index.js'; -import { ACTOR_PYTHON_EXAMPLE, DEFAULT_TOOL_NAMES } from '../const.js'; +import { addTool } from '../../src/tools/helpers.js'; +import { getActorsAsTools } from '../../src/tools/index.js'; +import type { Input } from '../../src/types.js'; +import { loadToolsFromInput } from '../../src/utils/tools-loader.js'; +import { ACTOR_PYTHON_EXAMPLE } from '../const.js'; import { expectArrayWeakEquals } from '../helpers.js'; beforeAll(() => { @@ -15,8 +17,11 @@ beforeAll(() => { describe('MCP server internals integration tests', () => { it('should load and restore tools from a tool list', async () => { - const actorsMcpServer = new ActorsMcpServer({ enableDefaultActors: true, enableAddingActors: true }, false); - await actorsMcpServer.initialize(); + const actorsMcpServer = new ActorsMcpServer(false); + const initialTools = await loadToolsFromInput({ + enableAddingActors: true, + } as Input, process.env.APIFY_TOKEN as string); + actorsMcpServer.upsertTools(initialTools); // Load new tool const newTool = await getActorsAsTools([ACTOR_PYTHON_EXAMPLE], process.env.APIFY_TOKEN as string); @@ -24,11 +29,10 @@ describe('MCP server internals integration tests', () => { // Store the tool name list const names = actorsMcpServer.listAllToolNames(); + // With enableAddingActors=true and no tools/actors, we should only have add-actor initially const expectedToolNames = [ - ...DEFAULT_TOOL_NAMES, - ...defaults.actors, - ...addRemoveTools.map((tool) => tool.tool.name), - ...[ACTOR_PYTHON_EXAMPLE], + addTool.tool.name, + ACTOR_PYTHON_EXAMPLE, ]; expectArrayWeakEquals(expectedToolNames, names); @@ -43,32 +47,10 @@ describe('MCP server internals integration tests', () => { expectArrayWeakEquals(actorsMcpServer.listAllToolNames(), expectedToolNames); }); - it('should reset and restore tool state with default tools', async () => { - const actorsMCPServer = new ActorsMcpServer({ enableDefaultActors: true, enableAddingActors: true }, false); - await actorsMCPServer.initialize(); - - const numberOfTools = defaultTools.length + addRemoveTools.length + defaults.actors.length; - const toolList = actorsMCPServer.listAllToolNames(); - expect(toolList.length).toEqual(numberOfTools); - // Add a new Actor - const newTool = await getActorsAsTools([ACTOR_PYTHON_EXAMPLE], process.env.APIFY_TOKEN as string); - actorsMCPServer.upsertTools(newTool); - - // Store the tool name list - const toolListWithActor = actorsMCPServer.listAllToolNames(); - expect(toolListWithActor.length).toEqual(numberOfTools + 1); // + 1 for the added Actor - - // Remove all tools - await actorsMCPServer.reset(); - // We connect second client so that the default tools are loaded - // if no specific list of Actors is provided - const toolListAfterReset = actorsMCPServer.listAllToolNames(); - expect(toolListAfterReset.length).toEqual(numberOfTools); - }); - it('should notify tools changed handler on tool modifications', async () => { let latestTools: string[] = []; - const numberOfTools = defaultTools.length + addRemoveTools.length + defaults.actors.length; + // With enableAddingActors=true and no tools/actors, seeded set contains only add-actor + const numberOfTools = 1; let toolNotificationCount = 0; const onToolsChanged = (tools: string[]) => { @@ -76,8 +58,9 @@ describe('MCP server internals integration tests', () => { toolNotificationCount++; }; - const actorsMCPServer = new ActorsMcpServer({ enableDefaultActors: true, enableAddingActors: true }, false); - await actorsMCPServer.initialize(); + const actorsMCPServer = new ActorsMcpServer(false); + const seeded = await loadToolsFromInput({ enableAddingActors: true } as Input, process.env.APIFY_TOKEN as string); + actorsMCPServer.upsertTools(seeded); actorsMCPServer.registerToolsChangedHandler(onToolsChanged); // Add a new Actor @@ -89,12 +72,8 @@ describe('MCP server internals integration tests', () => { expect(toolNotificationCount).toBe(1); expect(latestTools.length).toBe(numberOfTools + 1); expect(latestTools).toContain(actor); - for (const tool of [...defaultTools, ...addRemoveTools]) { - expect(latestTools).toContain(tool.tool.name); - } - for (const tool of defaults.actors) { - expect(latestTools).toContain(tool); - } + expect(latestTools).toContain(addTool.tool.name); + // No default actors are present when only add-actor is enabled by default // Remove the Actor actorsMCPServer.removeToolsByName([actorNameToToolName(actor)], true); @@ -103,25 +82,22 @@ describe('MCP server internals integration tests', () => { expect(toolNotificationCount).toBe(2); expect(latestTools.length).toBe(numberOfTools); expect(latestTools).not.toContain(actor); - for (const tool of [...defaultTools, ...addRemoveTools]) { - expect(latestTools).toContain(tool.tool.name); - } - for (const tool of defaults.actors) { - expect(latestTools).toContain(tool); - } + expect(latestTools).toContain(addTool.tool.name); + // No default actors are present by default in this mode }); it('should stop notifying after unregistering tools changed handler', async () => { let latestTools: string[] = []; let notificationCount = 0; - const numberOfTools = defaultTools.length + addRemoveTools.length + defaults.actors.length; + const numberOfTools = 1; const onToolsChanged = (tools: string[]) => { latestTools = tools; notificationCount++; }; - const actorsMCPServer = new ActorsMcpServer({ enableDefaultActors: true, enableAddingActors: true }, false); - await actorsMCPServer.initialize(); + const actorsMCPServer = new ActorsMcpServer(false); + const seeded = await loadToolsFromInput({ enableAddingActors: true } as Input, process.env.APIFY_TOKEN as string); + actorsMCPServer.upsertTools(seeded); actorsMCPServer.registerToolsChangedHandler(onToolsChanged); // Add a new Actor diff --git a/tests/integration/suite.ts b/tests/integration/suite.ts index 953762f0..bad0a526 100644 --- a/tests/integration/suite.ts +++ b/tests/integration/suite.ts @@ -4,10 +4,11 @@ import { ToolListChangedNotificationSchema } from '@modelcontextprotocol/sdk/typ import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it } from 'vitest'; import { defaults, HelperTools } from '../../src/const.js'; -import { latestNewsOnTopicPrompt } from '../../src/prompts/latest-news-on-topic.js'; -import { addRemoveTools, defaultTools, toolCategories, toolCategoriesEnabledByDefault } from '../../src/tools/index.js'; +import { addTool } from '../../src/tools/helpers.js'; +import { defaultTools, toolCategories } from '../../src/tools/index.js'; import { actorNameToToolName } from '../../src/tools/utils.js'; import type { ToolCategory } from '../../src/types.js'; +import { getExpectedToolNamesByCategories } from '../../src/utils/tools.js'; import { ACTOR_MCP_SERVER_ACTOR_NAME, ACTOR_PYTHON_EXAMPLE, DEFAULT_ACTOR_NAMES, DEFAULT_TOOL_NAMES } from '../const.js'; import { addActor, type McpClientOptions } from '../helpers.js'; @@ -87,27 +88,43 @@ export function createIntegrationTestsSuite( it('should list all default tools and Actors', async () => { const client = await createClientFn(); const tools = await client.listTools(); - expect(tools.tools.length).toEqual(defaultTools.length + defaults.actors.length + addRemoveTools.length); + expect(tools.tools.length).toEqual(defaultTools.length + defaults.actors.length); const names = getToolNames(tools); expectToolNamesToContain(names, DEFAULT_TOOL_NAMES); expectToolNamesToContain(names, DEFAULT_ACTOR_NAMES); - expectToolNamesToContain(names, addRemoveTools.map((tool) => tool.tool.name)); await client.close(); }); - it('should list all default tools and Actors, with add/remove tools', async () => { + it('should match spec default: actors,docs,apify/rag-web-browser when no params provided', async () => { + const client = await createClientFn(); + const tools = await client.listTools(); + const names = getToolNames(tools); + + // Should be equivalent to tools=actors,docs,apify/rag-web-browser + const expectedActorsTools = ['fetch-actor-details', 'search-actors', 'call-actor']; + const expectedDocsTools = ['search-apify-docs', 'fetch-apify-docs']; + const expectedActors = ['apify-slash-rag-web-browser']; + + const expectedTotal = expectedActorsTools.concat(expectedDocsTools, expectedActors); + expect(names).toHaveLength(expectedTotal.length); + + expectedActorsTools.forEach((tool) => expect(names).toContain(tool)); + expectedDocsTools.forEach((tool) => expect(names).toContain(tool)); + expectedActors.forEach((actor) => expect(names).toContain(actor)); + + await client.close(); + }); + + it('should list only add-actor when enableAddingActors is true and no tools/actors are specified', async () => { const client = await createClientFn({ enableAddingActors: true }); const names = getToolNames(await client.listTools()); - expect(names.length).toEqual(defaultTools.length + defaults.actors.length + addRemoveTools.length); - - expectToolNamesToContain(names, DEFAULT_TOOL_NAMES); - expectToolNamesToContain(names, DEFAULT_ACTOR_NAMES); - expectToolNamesToContain(names, addRemoveTools.map((tool) => tool.tool.name)); + expect(names.length).toEqual(1); + expect(names).toContain(addTool.tool.name); await client.close(); }); - it('should list all default tools and Actors, without add/remove tools', async () => { + it('should list all default tools and Actors when enableAddingActors is false', async () => { const client = await createClientFn({ enableAddingActors: false }); const names = getToolNames(await client.listTools()); expect(names.length).toEqual(defaultTools.length + defaults.actors.length); @@ -117,57 +134,189 @@ export function createIntegrationTestsSuite( await client.close(); }); - it('should list all default tools and two loaded Actors', async () => { - const actors = ['apify/website-content-crawler', 'apify/instagram-scraper']; + it('should override enableAddingActors false with experimental tool category', async () => { + const client = await createClientFn({ enableAddingActors: false, tools: ['experimental'] }); + const names = getToolNames(await client.listTools()); + expect(names).toHaveLength(toolCategories.experimental.length); + expect(names).toContain(addTool.tool.name); + await client.close(); + }); + + it('should list two loaded Actors', async () => { + const actors = ['apify/python-example', 'apify/rag-web-browser']; const client = await createClientFn({ actors, enableAddingActors: false }); const names = getToolNames(await client.listTools()); - expect(names.length).toEqual(defaultTools.length + actors.length); - expectToolNamesToContain(names, DEFAULT_TOOL_NAMES); + expect(names.length).toEqual(actors.length); expectToolNamesToContain(names, actors.map((actor) => actorNameToToolName(actor))); await client.close(); }); + it('should load only specified actors when actors param is provided (no other tools)', async () => { + const actors = ['apify/python-example']; + const client = await createClientFn({ actors }); + const names = getToolNames(await client.listTools()); + + // Should only load the specified actor, no default tools or categories + expect(names.length).toEqual(actors.length); + expect(names).toContain(actorNameToToolName(actors[0])); + + // Should NOT include any default category tools + expect(names).not.toContain('search-actors'); + expect(names).not.toContain('fetch-actor-details'); + expect(names).not.toContain('call-actor'); + expect(names).not.toContain('search-apify-docs'); + expect(names).not.toContain('fetch-apify-docs'); + + await client.close(); + }); + + it('should not load any tools when enableAddingActors is true and tools param is empty', async () => { + const client = await createClientFn({ enableAddingActors: true, tools: [] }); + const names = getToolNames(await client.listTools()); + expect(names).toHaveLength(0); + await client.close(); + }); + + it('should not load any tools when enableAddingActors is true and actors param is empty', async () => { + const client = await createClientFn({ enableAddingActors: true, actors: [] }); + const names = getToolNames(await client.listTools()); + expect(names.length).toEqual(0); + await client.close(); + }); + + it('should not load any tools when enableAddingActors is false and no tools/actors are specified', async () => { + const client = await createClientFn({ enableAddingActors: false, tools: [], actors: [] }); + const names = getToolNames(await client.listTools()); + expect(names.length).toEqual(0); + await client.close(); + }); + + it('should load only specified Actors via tools selectors when actors param omitted', async () => { + const actors = ['apify/python-example']; + const client = await createClientFn({ tools: actors }); + const names = getToolNames(await client.listTools()); + // Only the Actor should be loaded + expect(names).toHaveLength(actors.length); + expect(names).toContain(actorNameToToolName(actors[0])); + await client.close(); + }); + + it('should treat selectors with slashes as Actor names', async () => { + const client = await createClientFn({ + tools: ['docs', 'apify/python-example'], + }); + const names = getToolNames(await client.listTools()); + + // Should include docs category + expect(names).toContain('search-apify-docs'); + expect(names).toContain('fetch-apify-docs'); + + // Should include actor (if it exists/is valid) + expect(names).toContain('apify-slash-python-example'); + + await client.close(); + }); + + it('should merge actors param into tools selectors (backward compatibility)', async () => { + const actors = ['apify/python-example']; + const categories = ['docs'] as ToolCategory[]; + const client = await createClientFn({ tools: categories, actors }); + const names = getToolNames(await client.listTools()); + const docsToolNames = getExpectedToolNamesByCategories(categories); + const expected = [...docsToolNames, actorNameToToolName(actors[0])]; + expect(names).toHaveLength(expected.length); + const containsExpected = expected.every((n) => names.includes(n)); + expect(containsExpected).toBe(true); + await client.close(); + }); + + it('should handle mixed categories and specific tools in tools param', async () => { + const client = await createClientFn({ + tools: ['docs', 'fetch-actor-details', 'add-actor'], + }); + const names = getToolNames(await client.listTools()); + + // Should include: docs category + specific tools + expect(names).toContain('search-apify-docs'); // from docs category + expect(names).toContain('fetch-apify-docs'); // from docs category + expect(names).toContain('fetch-actor-details'); // specific tool + expect(names).toContain('add-actor'); // specific tool + + // Should NOT include other actors category tools + expect(names).not.toContain('search-actors'); + expect(names).not.toContain('call-actor'); + + await client.close(); + }); + + it('should load only docs tools', async () => { + const categories = ['docs'] as ToolCategory[]; + const client = await createClientFn({ tools: categories, actors: [] }); + const names = getToolNames(await client.listTools()); + const expected = getExpectedToolNamesByCategories(categories); + expect(names.length).toEqual(expected.length); + expectToolNamesToContain(names, expected); + await client.close(); + }); + + it('should load only a specific tool when tools includes a tool name', async () => { + const client = await createClientFn({ tools: ['fetch-actor-details'], actors: [] }); + const names = getToolNames(await client.listTools()); + expect(names).toEqual(['fetch-actor-details']); + await client.close(); + }); + + it('should not load any tools when tools param is empty and actors omitted', async () => { + const client = await createClientFn({ tools: [] }); + const names = getToolNames(await client.listTools()); + expect(names.length).toEqual(0); + await client.close(); + }); + + it('should not load any internal tools when tools param is empty and use custom Actor if specified', async () => { + const client = await createClientFn({ tools: [], actors: [ACTOR_PYTHON_EXAMPLE] }); + const names = getToolNames(await client.listTools()); + expect(names.length).toEqual(1); + expect(names).toContain(actorNameToToolName(ACTOR_PYTHON_EXAMPLE)); + await client.close(); + }); + it('should add Actor dynamically and call it directly', async () => { const selectedToolName = actorNameToToolName(ACTOR_PYTHON_EXAMPLE); const client = await createClientFn({ enableAddingActors: true }); const names = getToolNames(await client.listTools()); - const numberOfTools = defaultTools.length + addRemoveTools.length + defaults.actors.length; - expect(names.length).toEqual(numberOfTools); - // Check that the Actor is not in the tools list + // Only the add tool should be added + expect(names).toHaveLength(1); + expect(names).toContain('add-actor'); expect(names).not.toContain(selectedToolName); // Add Actor dynamically await addActor(client, ACTOR_PYTHON_EXAMPLE); // Check if tools was added const namesAfterAdd = getToolNames(await client.listTools()); - expect(namesAfterAdd.length).toEqual(numberOfTools + 1); + expect(namesAfterAdd.length).toEqual(2); expect(namesAfterAdd).toContain(selectedToolName); await callPythonExampleActor(client, selectedToolName); await client.close(); }); - it('should add Actor dynamically and call it via generic call-actor tool', async () => { + it('should call Actor dynamically via generic call-actor tool without need to add it first', async () => { const selectedToolName = actorNameToToolName(ACTOR_PYTHON_EXAMPLE); - const client = await createClientFn({ enableAddingActors: true, tools: ['preview'] }); + const client = await createClientFn({ enableAddingActors: true, tools: ['actors'] }); const names = getToolNames(await client.listTools()); - const numberOfTools = defaultTools.length + addRemoveTools.length + defaults.actors.length + toolCategories.preview.length; - expect(names.length).toEqual(numberOfTools); + // Only the actors category and add-actor should be loaded + const numberOfTools = toolCategories.actors.length + 1; + expect(names).toHaveLength(numberOfTools); // Check that the Actor is not in the tools list expect(names).not.toContain(selectedToolName); - // Add Actor dynamically - await addActor(client, ACTOR_PYTHON_EXAMPLE); - - // Check if tools was added - const namesAfterAdd = getToolNames(await client.listTools()); - expect(namesAfterAdd.length).toEqual(numberOfTools + 1); - expect(namesAfterAdd).toContain(selectedToolName); const result = await client.callTool({ name: HelperTools.ACTOR_CALL, arguments: { actor: ACTOR_PYTHON_EXAMPLE, + step: 'call', input: { first_number: 1, second_number: 2, @@ -178,6 +327,10 @@ export function createIntegrationTestsSuite( expect(result).toEqual( { content: [ + { + text: expect.stringMatching(/^Actor finished with runId: .+, datasetId .+$/), + type: 'text', + }, { text: `{"sum":3,"first_number":1,"second_number":2}`, type: 'text', @@ -189,60 +342,33 @@ export function createIntegrationTestsSuite( await client.close(); }); - it('should not call Actor via call-actor tool if it is not added', async () => { - const selectedToolName = actorNameToToolName(ACTOR_PYTHON_EXAMPLE); - const client = await createClientFn({ enableAddingActors: true, tools: ['preview'] }); - const names = getToolNames(await client.listTools()); - const numberOfTools = defaultTools.length + addRemoveTools.length + defaults.actors.length + toolCategories.preview.length; - expect(names.length).toEqual(numberOfTools); - // Check that the Actor is not in the tools list - expect(names).not.toContain(selectedToolName); + it('should enforce two-step process for call-actor tool', async () => { + const client = await createClientFn({ tools: ['actors'] }); - const result = await client.callTool({ + // Step 1: Get info (should work) + const infoResult = await client.callTool({ name: HelperTools.ACTOR_CALL, arguments: { actor: ACTOR_PYTHON_EXAMPLE, - input: { - first_number: 1, - second_number: 2, - }, + step: 'info', }, }); - // TODO: make some more change-tolerant assertion, it's hard to verify text message result without exact match - expect(result).toEqual( - { - content: [ - { - text: "Actor 'apify/python-example' is not added. Add it with the 'add-actor' tool. Available Actors are: apify/rag-web-browser", - type: 'text', - }, - ], - }, - ); - - await client.close(); - }); + expect(infoResult.content).toBeDefined(); + const content = infoResult.content as { text: string }[]; + expect(content.some((item) => item.text.includes('Input Schema'))).toBe(true); - // TODO: disabled for now, remove tools is disabled and might be removed in the future - it.skip('should remove Actor from tools list', async () => { - const actor = ACTOR_PYTHON_EXAMPLE; - const selectedToolName = actorNameToToolName(actor); - const client = await createClientFn({ - actors: [actor], - enableAddingActors: true, + // Step 2: Call with proper input (should work) + const callResult = await client.callTool({ + name: HelperTools.ACTOR_CALL, + arguments: { + actor: ACTOR_PYTHON_EXAMPLE, + step: 'call', + input: { first_number: 1, second_number: 2 }, + }, }); - // Verify actor is in the tools list - const namesBefore = getToolNames(await client.listTools()); - expect(namesBefore).toContain(selectedToolName); - - // Remove the actor - await client.callTool({ name: HelperTools.ACTOR_REMOVE, arguments: { toolName: selectedToolName } }); - - // Verify actor is removed - const namesAfter = getToolNames(await client.listTools()); - expect(namesAfter).not.toContain(selectedToolName); + expect(callResult.content).toBeDefined(); await client.close(); }); @@ -331,14 +457,14 @@ export function createIntegrationTestsSuite( const toolNamesBefore = getToolNames(await client.listTools()); const searchToolCountBefore = toolNamesBefore.filter((name) => name.includes(HelperTools.STORE_SEARCH)).length; - expect(searchToolCountBefore).toBe(1); + expect(searchToolCountBefore).toBe(0); // Add self as an Actorized MCP server await addActor(client, ACTOR_MCP_SERVER_ACTOR_NAME); const toolNamesAfter = getToolNames(await client.listTools()); const searchToolCountAfter = toolNamesAfter.filter((name) => name.includes(HelperTools.STORE_SEARCH)).length; - expect(searchToolCountAfter).toBe(2); + expect(searchToolCountAfter).toBe(1); // Find the search tool from the Actorized MCP server const actorizedMCPSearchTool = toolNamesAfter.find( @@ -414,13 +540,8 @@ export function createIntegrationTestsSuite( const loadedTools = await client.listTools(); const toolNames = getToolNames(loadedTools); - // If the category is enabled by default, it should not be loaded again, and its tools - // are accounted for in the default tools. - const isCategoryInDefault = toolCategoriesEnabledByDefault.includes(category as ToolCategory); - const expectedTools = isCategoryInDefault ? [] : toolCategories[category as ToolCategory]; - const expectedToolNames = expectedTools.map((tool) => tool.tool.name); - - expect(toolNames.length).toEqual(expectedTools.length + defaultTools.length + defaults.actors.length + addRemoveTools.length); + const expectedToolNames = getExpectedToolNamesByCategories([category as ToolCategory]); + // Only assert that all tools from the selected category are present. for (const expectedToolName of expectedToolNames) { expect(toolNames).toContain(expectedToolName); } @@ -429,6 +550,35 @@ export function createIntegrationTestsSuite( } }); + it('should include add-actor when experimental category is selected even if enableAddingActors is false', async () => { + const client = await createClientFn({ + enableAddingActors: false, + tools: ['experimental'], + }); + + const loadedTools = await client.listTools(); + const toolNames = getToolNames(loadedTools); + + expect(toolNames).toContain(addTool.tool.name); + + await client.close(); + }); + + it('should include add-actor when enableAddingActors is false and add-actor is selected directly', async () => { + const client = await createClientFn({ + enableAddingActors: false, + tools: [addTool.tool.name], + }); + + const loadedTools = await client.listTools(); + const toolNames = getToolNames(loadedTools); + + // Must include add-actor since it was selected directly + expect(toolNames).toContain(addTool.tool.name); + + await client.close(); + }); + it('should handle multiple tool category keys input correctly', async () => { const categories = ['docs', 'runs', 'storage'] as ToolCategory[]; const client = await createClientFn({ @@ -438,25 +588,10 @@ export function createIntegrationTestsSuite( const loadedTools = await client.listTools(); const toolNames = getToolNames(loadedTools); - const expectedTools = [ - ...toolCategories.docs, - ...toolCategories.runs, - ...toolCategories.storage, - ]; - const expectedToolNames = expectedTools.map((tool) => tool.tool.name); - - // Handle case where tools are enabled by default - const selectedCategoriesInDefault = categories.filter((key) => toolCategoriesEnabledByDefault.includes(key)); - const numberOfToolsFromCategoriesInDefault = selectedCategoriesInDefault - .flatMap((key) => toolCategories[key]).length; - - const numberOfToolsExpected = defaultTools.length + defaults.actors.length + addRemoveTools.length - // Tools from tool categories minus the ones already in default tools - + (expectedTools.length - numberOfToolsFromCategoriesInDefault); - expect(toolNames.length).toEqual(numberOfToolsExpected); - for (const expectedToolName of expectedToolNames) { - expect(toolNames).toContain(expectedToolName); - } + const expectedToolNames = getExpectedToolNamesByCategories(categories); + expect(toolNames).toHaveLength(expectedToolNames.length); + const containsExpectedTools = toolNames.every((name) => expectedToolNames.includes(name)); + expect(containsExpectedTools).toBe(true); await client.close(); }); @@ -473,7 +608,7 @@ export function createIntegrationTestsSuite( const topic = 'apify'; const prompt = await client.getPrompt({ - name: latestNewsOnTopicPrompt.name, + name: 'GetLatestNewsOnTopic', arguments: { topic, }, @@ -499,10 +634,7 @@ export function createIntegrationTestsSuite( const actors = ['apify/python-example', 'apify/rag-web-browser']; const client = await createClientFn({ actors, useEnv: true }); const names = getToolNames(await client.listTools()); - expect(names.length).toEqual(defaultTools.length + actors.length + addRemoveTools.length); - expectToolNamesToContain(names, DEFAULT_TOOL_NAMES); expectToolNamesToContain(names, actors.map((actor) => actorNameToToolName(actor))); - expectToolNamesToContain(names, addRemoveTools.map((tool) => tool.tool.name)); await client.close(); }); @@ -518,6 +650,15 @@ export function createIntegrationTestsSuite( await client.close(); }); + it.runIf(options.transport === 'stdio')('should respect ENABLE_ADDING_ACTORS environment variable and load only add-actor tool when true', async () => { + // Test with enableAddingActors = false via env var + const client = await createClientFn({ enableAddingActors: true, useEnv: true }); + const names = getToolNames(await client.listTools()); + expect(names).toEqual(['add-actor']); + + await client.close(); + }); + it.runIf(options.transport === 'stdio')('should load tool categories from TOOLS environment variable', async () => { const categories = ['docs', 'runs'] as ToolCategory[]; const client = await createClientFn({ tools: categories, useEnv: true }); @@ -531,15 +672,7 @@ export function createIntegrationTestsSuite( ]; const expectedToolNames = expectedTools.map((tool) => tool.tool.name); - // Handle case where tools are enabled by default - const selectedCategoriesInDefault = categories.filter((key) => toolCategoriesEnabledByDefault.includes(key)); - const numberOfToolsFromCategoriesInDefault = selectedCategoriesInDefault - .flatMap((key) => toolCategories[key]).length; - - const numberOfToolsExpected = defaultTools.length + defaults.actors.length + addRemoveTools.length - // Tools from tool categories minus the ones already in default tools - + (expectedTools.length - numberOfToolsFromCategoriesInDefault); - expect(toolNames.length).toEqual(numberOfToolsExpected); + expect(toolNames).toHaveLength(expectedToolNames.length); for (const expectedToolName of expectedToolNames) { expect(toolNames).toContain(expectedToolName); } diff --git a/tests/unit/input.test.ts b/tests/unit/input.test.ts index 61ce0068..86996e04 100644 --- a/tests/unit/input.test.ts +++ b/tests/unit/input.test.ts @@ -1,23 +1,218 @@ import { describe, expect, it } from 'vitest'; -import { processInput } from '../../src/input.js'; +import { normalizeList, processInput, toBoolean } from '../../src/input.js'; import type { Input } from '../../src/types.js'; +describe('toBoolean', () => { + describe('with defaultValue true', () => { + it('should return true for undefined value', () => { + expect(toBoolean(undefined, true)).toBe(true); + }); + + it('should return true for boolean true', () => { + expect(toBoolean(true, true)).toBe(true); + }); + + it('should return false for boolean false', () => { + expect(toBoolean(false, true)).toBe(false); + }); + + it('should return true for string "true" (case insensitive), regardless of default', () => { + // Using default=false to prove the function returns true, not the default + expect(toBoolean('true', false)).toBe(true); + expect(toBoolean('TRUE', false)).toBe(true); + expect(toBoolean('True', false)).toBe(true); + expect(toBoolean('TrUe', false)).toBe(true); + }); + + it('should return false for string "false" (case insensitive), regardless of default', () => { + // Using default=true to prove the function returns false, not the default + expect(toBoolean('false', true)).toBe(false); + expect(toBoolean('FALSE', true)).toBe(false); + expect(toBoolean('False', true)).toBe(false); + expect(toBoolean('FaLsE', true)).toBe(false); + }); + + it('should return false for non-boolean strings (not "true"), regardless of default', () => { + // Using default=true to prove the function returns false, not the default + expect(toBoolean('yes', true)).toBe(false); + expect(toBoolean('no', true)).toBe(false); + expect(toBoolean('1', true)).toBe(false); + expect(toBoolean('0', true)).toBe(false); + expect(toBoolean('', true)).toBe(false); + expect(toBoolean('random', true)).toBe(false); + }); + + it('should return default value for non-string, non-boolean types', () => { + expect(toBoolean(1, true)).toBe(true); + expect(toBoolean(0, true)).toBe(true); + expect(toBoolean(null, true)).toBe(true); + expect(toBoolean({}, true)).toBe(true); + expect(toBoolean([], true)).toBe(true); + }); + + it('should demonstrate default value behavior with opposite defaults', () => { + // Same input, different defaults - proves it uses the default for non-string/non-boolean + expect(toBoolean(1, true)).toBe(true); + expect(toBoolean(1, false)).toBe(false); + expect(toBoolean(null, true)).toBe(true); + expect(toBoolean(null, false)).toBe(false); + expect(toBoolean({}, true)).toBe(true); + expect(toBoolean({}, false)).toBe(false); + }); + }); + + describe('with defaultValue false', () => { + it('should return false for undefined value', () => { + expect(toBoolean(undefined, false)).toBe(false); + }); + + it('should return true for boolean true', () => { + expect(toBoolean(true, false)).toBe(true); + }); + + it('should return false for boolean false', () => { + expect(toBoolean(false, false)).toBe(false); + }); + + it('should return true for string "true" (case insensitive), regardless of default', () => { + // Using default=false to prove the function returns true, not the default + expect(toBoolean('true', false)).toBe(true); + expect(toBoolean('TRUE', false)).toBe(true); + expect(toBoolean('True', false)).toBe(true); + }); + + it('should return false for string "false" (case insensitive), regardless of default', () => { + // Using default=true to prove the function returns false, not the default + expect(toBoolean('false', true)).toBe(false); + expect(toBoolean('FALSE', true)).toBe(false); + expect(toBoolean('False', true)).toBe(false); + }); + + it('should return false for non-boolean strings (not "true"), regardless of default', () => { + // Using default=true to prove the function returns false, not the default + expect(toBoolean('yes', true)).toBe(false); + expect(toBoolean('no', true)).toBe(false); + expect(toBoolean('1', true)).toBe(false); + expect(toBoolean('0', true)).toBe(false); + expect(toBoolean('', true)).toBe(false); + expect(toBoolean('random', true)).toBe(false); + }); + + it('should return default value for non-string, non-boolean types', () => { + expect(toBoolean(1, false)).toBe(false); + expect(toBoolean(0, false)).toBe(false); + expect(toBoolean(null, false)).toBe(false); + expect(toBoolean({}, false)).toBe(false); + expect(toBoolean([], false)).toBe(false); + }); + }); +}); + +describe('normalizeList', () => { + describe('undefined input', () => { + it('should return undefined for undefined input', () => { + expect(normalizeList(undefined)).toBeUndefined(); + }); + }); + + describe('array input', () => { + it('should return trimmed array for string array', () => { + expect(normalizeList(['item1', 'item2', 'item3'])).toEqual(['item1', 'item2', 'item3']); + }); + + it('should trim whitespace from array items', () => { + expect(normalizeList([' item1 ', ' item2 ', 'item3\t'])).toEqual(['item1', 'item2', 'item3']); + }); + + it('should filter out empty strings from array', () => { + expect(normalizeList(['item1', '', 'item2', ' ', 'item3'])).toEqual(['item1', 'item2', 'item3']); + }); + + it('should convert non-string array items to strings', () => { + expect(normalizeList([1, 2, 'item3'] as (string | number)[])).toEqual(['1', '2', 'item3']); + }); + + it('should handle empty array', () => { + expect(normalizeList([])).toEqual([]); + }); + + it('should handle array with only empty/whitespace strings', () => { + expect(normalizeList(['', ' ', '\t', '\n'])).toEqual([]); + }); + }); + + describe('string input', () => { + it('should split comma-separated string', () => { + expect(normalizeList('item1,item2,item3')).toEqual(['item1', 'item2', 'item3']); + }); + + it('should trim whitespace around commas', () => { + expect(normalizeList('item1, item2 , item3')).toEqual(['item1', 'item2', 'item3']); + }); + + it('should handle extra whitespace and commas', () => { + expect(normalizeList(' item1 , , item2 , item3 ')).toEqual(['item1', 'item2', 'item3']); + }); + + it('should return empty array for empty string', () => { + expect(normalizeList('')).toEqual([]); + }); + + it('should return empty array for whitespace-only string', () => { + expect(normalizeList(' ')).toEqual([]); + expect(normalizeList('\t\n')).toEqual([]); + }); + + it('should handle single item without commas', () => { + expect(normalizeList('single-item')).toEqual(['single-item']); + }); + + it('should handle string with only commas', () => { + expect(normalizeList(',,,,')).toEqual([]); + }); + + it('should handle mixed empty and valid items', () => { + expect(normalizeList('item1,,item2, ,item3')).toEqual(['item1', 'item2', 'item3']); + }); + + it('should handle trailing and leading commas', () => { + expect(normalizeList(',item1,item2,item3,')).toEqual(['item1', 'item2', 'item3']); + }); + }); + + describe('edge cases', () => { + it('should handle numeric string input', () => { + expect(normalizeList('1,2,3')).toEqual(['1', '2', '3']); + }); + + it('should handle special characters in items', () => { + expect(normalizeList('item@1,item#2,item$3')).toEqual(['item@1', 'item#2', 'item$3']); + }); + + it('should handle items with internal spaces', () => { + expect(normalizeList('item one,item two,item three')).toEqual(['item one', 'item two', 'item three']); + }); + }); +}); + describe('processInput', () => { - it('should handle string actors input and convert to array', async () => { + it('should handle string actors input and convert to tools', async () => { const input: Partial = { actors: 'actor1, actor2,actor3', }; const processed = processInput(input); - expect(processed.actors).toEqual(['actor1', 'actor2', 'actor3']); + expect(processed.tools).toEqual(['actor1', 'actor2', 'actor3']); + expect(processed.actors).toBeUndefined(); }); - it('should keep array actors input unchanged', async () => { + it('should move array actors input into tools', async () => { const input: Partial = { actors: ['actor1', 'actor2', 'actor3'], }; const processed = processInput(input); - expect(processed.actors).toEqual(['actor1', 'actor2', 'actor3']); + expect(processed.tools).toEqual(['actor1', 'actor2', 'actor3']); + expect(processed.actors).toBeUndefined(); }); it('should handle enableActorAutoLoading to set enableAddingActors', async () => { @@ -39,17 +234,14 @@ describe('processInput', () => { expect(processed.enableAddingActors).toBe(false); }); - it('should default enableAddingActors to true when not provided', async () => { - const input: Partial = { - actors: ['actor1'], - }; + it('should default enableAddingActors to false when not provided', async () => { + const input: Partial = { }; const processed = processInput(input); - expect(processed.enableAddingActors).toBe(true); + expect(processed.enableAddingActors).toBe(false); }); it('should keep tools as array of valid featureTools keys', async () => { const input: Partial = { - actors: ['actor1'], tools: ['docs', 'runs'], }; const processed = processInput(input); @@ -58,28 +250,60 @@ describe('processInput', () => { it('should handle empty tools array', async () => { const input: Partial = { - actors: ['actor1'], tools: [], }; const processed = processInput(input); expect(processed.tools).toEqual([]); }); - it('should handle missing tools field (undefined)', async () => { + it('should handle missing tools field (undefined) by moving actors into tools', async () => { const input: Partial = { actors: ['actor1'], }; const processed = processInput(input); - expect(processed.tools).toBeUndefined(); + expect(processed.tools).toEqual(['actor1']); + expect(processed.actors).toBeUndefined(); }); it('should include all keys, even invalid ones', async () => { const input: Partial = { - actors: ['actor1'], - // @ts-expect-error: purposely invalid key for test tools: ['docs', 'invalidKey', 'storage'], }; const processed = processInput(input); expect(processed.tools).toEqual(['docs', 'invalidKey', 'storage']); }); + + it('should merge actors into tools for backward compatibility', async () => { + const input: Partial = { + actors: ['apify/website-content-crawler', 'apify/instagram-scraper'], + tools: ['docs'], + }; + const processed = processInput(input); + expect(processed.tools).toEqual([ + 'docs', + 'apify/website-content-crawler', + 'apify/instagram-scraper', + ]); + }); + + it('should merge actors into tools when tools is a string', async () => { + const input: Partial = { + actors: ['apify/instagram-scraper'], + tools: 'runs', + }; + const processed = processInput(input); + expect(processed.tools).toEqual([ + 'runs', + 'apify/instagram-scraper', + ]); + }); + + it('should not modify tools if actors is empty array', async () => { + const input: Partial = { + actors: [], + tools: ['docs'], + }; + const processed = processInput(input); + expect(processed.tools).toEqual(['docs']); + }); }); diff --git a/tests/unit/mcp.utils.test.ts b/tests/unit/mcp.utils.test.ts index 10fc8c6c..c81287f7 100644 --- a/tests/unit/mcp.utils.test.ts +++ b/tests/unit/mcp.utils.test.ts @@ -3,45 +3,48 @@ import { describe, expect, it } from 'vitest'; import { parseInputParamsFromUrl } from '../../src/mcp/utils.js'; describe('parseInputParamsFromUrl', () => { - it('should parse Actors from URL query params', () => { - const url = 'https://actors-mcp-server.apify.actor?token=123&actors=apify/web-scraper'; + it('should parse Actors from URL query params (as tools)', () => { + const url = 'https://mcp.apify.com?token=123&actors=apify/web-scraper'; const result = parseInputParamsFromUrl(url); - expect(result.actors).toEqual(['apify/web-scraper']); + expect(result.tools).toEqual(['apify/web-scraper']); + expect(result.actors).toBeUndefined(); }); - it('should parse multiple Actors from URL', () => { - const url = 'https://actors-mcp-server.apify.actor?actors=apify/instagram-scraper,lukaskrivka/google-maps'; + it('should parse multiple Actors from URL (as tools)', () => { + const url = 'https://mcp.apify.com?actors=apify/instagram-scraper,lukaskrivka/google-maps'; const result = parseInputParamsFromUrl(url); - expect(result.actors).toEqual(['apify/instagram-scraper', 'lukaskrivka/google-maps']); + expect(result.tools).toEqual(['apify/instagram-scraper', 'lukaskrivka/google-maps']); + expect(result.actors).toBeUndefined(); }); it('should handle URL without query params', () => { - const url = 'https://actors-mcp-server.apify.actor'; + const url = 'https://mcp.apify.com'; const result = parseInputParamsFromUrl(url); expect(result.actors).toBeUndefined(); }); it('should parse enableActorAutoLoading flag', () => { - const url = 'https://actors-mcp-server.apify.actor?enableActorAutoLoading=true'; + const url = 'https://mcp.apify.com?enableActorAutoLoading=true'; const result = parseInputParamsFromUrl(url); expect(result.enableAddingActors).toBe(true); }); it('should parse enableAddingActors flag', () => { - const url = 'https://actors-mcp-server.apify.actor?enableAddingActors=true'; + const url = 'https://mcp.apify.com?enableAddingActors=true'; const result = parseInputParamsFromUrl(url); expect(result.enableAddingActors).toBe(true); }); it('should parse enableAddingActors flag', () => { - const url = 'https://actors-mcp-server.apify.actor?enableAddingActors=false'; + const url = 'https://mcp.apify.com?enableAddingActors=false'; const result = parseInputParamsFromUrl(url); expect(result.enableAddingActors).toBe(false); }); - it('should handle Actors as string parameter', () => { - const url = 'https://actors-mcp-server.apify.actor?actors=apify/rag-web-browser'; + it('should handle Actors as string parameter (as tools)', () => { + const url = 'https://mcp.apify.com?actors=apify/rag-web-browser'; const result = parseInputParamsFromUrl(url); - expect(result.actors).toEqual(['apify/rag-web-browser']); + expect(result.tools).toEqual(['apify/rag-web-browser']); + expect(result.actors).toBeUndefined(); }); });