Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 21 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ For example it can:
- use [Instagram Scraper](https://apify.com/apify/instagram-scraper) to scrape Instagram posts, profiles, places, photos, and comments
- use [RAG Web Browser](https://apify.com/apify/web-scraper) to search the web, scrape the top N URLs, and return their content

To interact with the Apify MCP server, you can use MCP clients such as [Claude Desktop](https://claude.ai/download), [Superinference.ai](https://superinterface.ai/), or [LibreChat](https://www.librechat.ai/).

To interact with the Apify MCP server, you can use MCP clients such as [Claude Desktop](https://claude.ai/download), [LibreChat](https://www.librechat.ai/), or other [MCP clients](https://glama.ai/mcp/clients).
Additionally, you can use simple example clients found in the [examples](https://github.com/apify/actor-mcp-server/tree/main/src/examples) directory.

When you have Actors integrated with the MCP server, you can ask:
Expand All @@ -29,6 +30,10 @@ When you have Actors integrated with the MCP server, you can ask:
- "Provide a step-by-step guide on using the Model Context Protocol with source URLs."
- "What Apify Actors I can use?"

The following image shows how the Apify MCP server interacts with the Apify platform and AI clients:

![Actors-MCP-server](https://raw.githubusercontent.com/apify/actors-mcp-server/refs/heads/master/docs/actors-mcp-server.png)

In the future, we plan to load Actors dynamically and provide Apify's dataset and key-value store as resources.
See the [Roadmap](#-roadmap-january-2025) for more details.

Expand Down Expand Up @@ -301,15 +306,27 @@ npm run build
You can launch the MCP Inspector via [`npm`](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm) with this command:

```bash
npx @modelcontextprotocol/inspector node /path/to/actor-mcp-server/dist/index.js --env APIFY_TOKEN=your-apify-token
npx @modelcontextprotocol/inspector node @apify/actors-mcp-server --env APIFY_TOKEN=your-apify-token
```

Upon launching, the Inspector will display a URL that you can access in your browser to begin debugging.

## ⓘ Limitations and feedback

To limit the context size the properties in the `input schema` are pruned and description is truncated to 200 characters.
Enum fields and titles are truncated to max 50 options.

Memory for each Actor is limited to 4GB.
Free users have an 8GB limit, 128MB needs to be allocated for running `Actors-MCP-Server`.

If you need other features or have any feedback, please [submit an issue](https://console.apify.com/actors/3ox4R101TgZz67sLr/issues) in Apify Console to let us know.

Are you interested in AI Agents and AI applications?
Visit the [Model Context Protocol](https://modelcontextprotocol.org/) website and read blog post [What are AI agents?](https://blog.apify.com/what-are-ai-agents/).

# 🚀 Roadmap (January 2025)

- Document examples for [Superinference.ai](https://superinterface.ai/) and [LibreChat](https://www.librechat.ai/).
- Document examples for [LibreChat](https://www.librechat.ai/).
- Provide tools to search for Actors and load them as needed.
- Add Apify's dataset and key-value store as resources.
- Add tools such as Actor logs and Actor runs for debugging.
- Prune Actors input schema to reduce context size.
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@apify/actors-mcp-server",
"version": "0.1.1",
"version": "0.1.2",
"type": "module",
"description": "Model Context Protocol Server for Apify Actors",
"engines": {
Expand Down
42 changes: 41 additions & 1 deletion src/actorDefinition.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
import { Ajv } from 'ajv';
import { ApifyClient } from 'apify-client';

import { MAX_DESCRIPTION_LENGTH, MAX_ENUM_LENGTH, MAX_MEMORY_MBYTES } from './const.js';
import { log } from './logger.js';
import type { ActorDefinitionWithDesc, Tool } from './types';
import type { ActorDefinitionWithDesc, SchemaProperties, Tool } from './types.js';

/**
* Get actor input schema by actor name.
Expand Down Expand Up @@ -43,6 +44,7 @@ async function fetchActorDefinition(actorFullName: string): Promise<ActorDefinit
const actorDefinitions = buildDetails?.actorDefinition as ActorDefinitionWithDesc;
actorDefinitions.description = actor.description || '';
actorDefinitions.name = actorFullName;
actorDefinitions.defaultRunOptions = actor.defaultRunOptions;
return actorDefinitions;
}
return null;
Expand All @@ -52,6 +54,38 @@ async function fetchActorDefinition(actorFullName: string): Promise<ActorDefinit
}
}

/**
* Shortens the description and enum values of schema properties.
* @param properties
*/
function shortenProperties(properties: { [key: string]: SchemaProperties}): { [key: string]: SchemaProperties } {
for (const property of Object.values(properties)) {
if (property.description.length > MAX_DESCRIPTION_LENGTH) {
property.description = `${property.description.slice(0, MAX_DESCRIPTION_LENGTH)}...`;
}
if (property.enum) {
property.enum = property.enum.slice(0, MAX_ENUM_LENGTH);
}
if (property.enumTitles) {
property.enumTitles = property.enumTitles.slice(0, MAX_ENUM_LENGTH);
}
}
return properties;
}

/**
* Filters schema properties to include only the necessary fields.
* @param properties
*/
function filterSchemaProperties(properties: { [key: string]: SchemaProperties }): { [key: string]: SchemaProperties } {
const filteredProperties: { [key: string]: SchemaProperties } = {};
for (const [key, property] of Object.entries(properties)) {
const { title, description, enum: enumValues, enumTitles, type, default: defaultValue, prefill } = property;
filteredProperties[key] = { title, description, enum: enumValues, enumTitles, type, default: defaultValue, prefill };
}
return filteredProperties;
}

/**
* Fetches actor input schemas by actor full names and creates MCP tools.
*
Expand All @@ -70,13 +104,19 @@ export async function getActorsAsTools(actors: string[]): Promise<Tool[]> {
const tools = [];
for (const result of results) {
if (result) {
if (result.input && 'properties' in result.input && result.input) {
const properties = filterSchemaProperties(result.input.properties as { [key: string]: SchemaProperties });
result.input.properties = shortenProperties(properties);
}
try {
const memoryMbytes = result.defaultRunOptions?.memoryMbytes || MAX_MEMORY_MBYTES;
tools.push({
name: result.name.replace('/', '_'),
actorName: result.name,
description: result.description,
inputSchema: result.input || {},
ajvValidate: ajv.compile(result.input || {}),
memoryMbytes: memoryMbytes > MAX_MEMORY_MBYTES ? MAX_MEMORY_MBYTES : memoryMbytes,
});
} catch (validationError) {
log.error(`Failed to compile AJV schema for actor: ${result.name}. Error: ${validationError}`);
Expand Down
9 changes: 9 additions & 0 deletions src/const.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,15 @@
export const SERVER_NAME = 'apify-mcp-server';
export const SERVER_VERSION = '0.1.0';

export const HEADER_READINESS_PROBE = 'x-apify-container-server-readiness-probe';

export const MAX_ENUM_LENGTH = 50;
export const MAX_DESCRIPTION_LENGTH = 200;
// Limit memory to 4GB for Actors. Free users have 8 GB limit, but we need to reserve some memory for Actors-MCP-Server too
export const MAX_MEMORY_MBYTES = 4096;

export const USER_AGENT_ORIGIN = 'Origin/mcp-server';

export const defaults = {
actors: [
'apify/instagram-scraper',
Expand Down
7 changes: 6 additions & 1 deletion src/examples/client_sse.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,18 @@ async def run() -> None:

tools = await session.list_tools()
print("Available Tools:", tools, end="\n\n")
for tool in tools.tools:
print(f"\n### Tool name ###: {tool.name}")
print(f"\tdescription: {tool.description}")
print(f"\tinputSchema: {tool.inputSchema}")

if hasattr(tools, "tools") and not tools.tools:
print("No tools available!")
return

print("\n\nCall tool")
result = await session.call_tool("apify/rag-web-browser", { "query": "example.com", "maxResults": 3 })
print("Tools Call Result:")
print("Tools call result:")

for content in result.content:
print(content)
Expand Down
11 changes: 9 additions & 2 deletions src/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,11 @@ import { parse } from 'querystring';

import { SSEServerTransport } from '@modelcontextprotocol/sdk/server/sse.js';
import { Actor } from 'apify';
import type { ActorCallOptions } from 'apify-client';
import type { Request, Response } from 'express';
import express from 'express';

import { Routes } from './const.js';
import { HEADER_READINESS_PROBE, MAX_MEMORY_MBYTES, Routes } from './const.js';
import { processInput } from './input.js';
import { log } from './logger.js';
import { ApifyMcpServer } from './server.js';
Expand Down Expand Up @@ -46,6 +47,11 @@ async function processParamsAndUpdateTools(url: string) {

app.route(Routes.ROOT)
.get(async (req: Request, res: Response) => {
if (req.headers && req.get(HEADER_READINESS_PROBE) !== undefined) {
log.debug('Received readiness probe');
res.status(200).json({ message: 'Server is ready' }).end();
return;
}
try {
log.info(`Received GET message at: ${req.url}`);
await processParamsAndUpdateTools(req.url);
Expand Down Expand Up @@ -110,6 +116,7 @@ if (STANDBY_MODE) {
if (input && !input.debugActor && !input.debugActorInput) {
await Actor.fail('If you need to debug a specific actor, please provide the debugActor and debugActorInput fields in the input');
}
await mcpServer.callActorGetDataset(input.debugActor!, input.debugActorInput!);
const options = { memory: MAX_MEMORY_MBYTES } as ActorCallOptions;
await mcpServer.callActorGetDataset(input.debugActor!, input.debugActorInput!, options);
await Actor.exit();
}
31 changes: 27 additions & 4 deletions src/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,11 @@
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
import type { Transport } from '@modelcontextprotocol/sdk/shared/transport.js';
import { CallToolRequestSchema, ListToolsRequestSchema } from '@modelcontextprotocol/sdk/types.js';
import type { ApifyClientOptions } from 'apify';
import { Actor } from 'apify';
import type { ActorCallOptions } from 'apify-client';
import { ApifyClient } from 'apify-client';
import type { AxiosRequestConfig } from 'axios';

import { getActorsAsTools } from './actorDefinition.js';
import {
Expand All @@ -15,6 +18,7 @@ import {
defaults,
SERVER_NAME,
SERVER_VERSION,
USER_AGENT_ORIGIN,
} from './const.js';
import { log } from './logger.js';
import type { Tool } from './types';
Expand Down Expand Up @@ -43,27 +47,46 @@ export class ApifyMcpServer {
this.setupToolHandlers();
}

/**
* Adds a User-Agent header to the request config.
* @param config
* @private
*/
private addUserAgent(config: AxiosRequestConfig): AxiosRequestConfig {
const updatedConfig = { ...config };
updatedConfig.headers = updatedConfig.headers ?? {};
updatedConfig.headers['User-Agent'] = `${updatedConfig.headers['User-Agent'] ?? ''}; ${USER_AGENT_ORIGIN}`;
return updatedConfig;
}

/**
* Calls an Apify actor and retrieves the dataset items.
*
* It requires the `APIFY_TOKEN` environment variable to be set.
* If the `APIFY_IS_AT_HOME` the dataset items are pushed to the Apify dataset.
*
* @param {string} actorName - The name of the actor to call.
* @param {ActorCallOptions} callOptions - The options to pass to the actor.
* @param {unknown} input - The input to pass to the actor.
* @returns {Promise<object[]>} - A promise that resolves to an array of dataset items.
* @throws {Error} - Throws an error if the `APIFY_TOKEN` is not set
*/
public async callActorGetDataset(actorName: string, input: unknown): Promise<object[]> {
public async callActorGetDataset(
actorName: string,
input: unknown,
callOptions: ActorCallOptions | undefined = undefined,
): Promise<object[]> {
if (!process.env.APIFY_TOKEN) {
throw new Error('APIFY_TOKEN is required but not set. Please set it as an environment variable');
}
try {
log.info(`Calling actor ${actorName} with input: ${JSON.stringify(input)}`);
const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const options: ApifyClientOptions = { requestInterceptors: [this.addUserAgent] };
const client = new ApifyClient({ ...options, token: process.env.APIFY_TOKEN });
const actorClient = client.actor(actorName);

const results = await actorClient.call(input);
const results = await actorClient.call(input, callOptions);
const dataset = await client.dataset(results.defaultDatasetId).listItems();
log.info(`Actor ${actorName} finished with ${dataset.items.length} items`);

Expand Down Expand Up @@ -132,7 +155,7 @@ export class ApifyMcpServer {
}

try {
const items = await this.callActorGetDataset(tool.actorName, args);
const items = await this.callActorGetDataset(tool.actorName, args, { memory: tool.memoryMbytes } as ActorCallOptions);
const content = items.map((item) => {
const text = JSON.stringify(item).slice(0, ACTOR_OUTPUT_MAX_CHARS_PER_ITEM);
return text.length === ACTOR_OUTPUT_MAX_CHARS_PER_ITEM
Expand Down
14 changes: 13 additions & 1 deletion src/types.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import type { ValidateFunction } from 'ajv';
import type { ActorDefinition } from 'apify-client';
import type { ActorDefaultRunOptions, ActorDefinition } from 'apify-client';

export type Input = {
actors: string[] | string;
Expand All @@ -9,6 +9,7 @@ export type Input = {

export interface ActorDefinitionWithDesc extends ActorDefinition {
description: string;
defaultRunOptions: ActorDefaultRunOptions
}

export interface Tool {
Expand All @@ -17,4 +18,15 @@ export interface Tool {
description: string;
inputSchema: object;
ajvValidate: ValidateFunction;
memoryMbytes: number;
}

export interface SchemaProperties {
title: string;
description: string;
enum: string[]; // Array of string options for the enum
enumTitles: string[]; // Array of string titles for the enum
type: string; // Data type (e.g., "string")
default: string;
prefill: string;
}
Loading