feat: Decouple Actor from mcp-server, add ability to call Actorized MCP and load tools #59

MQ37 · 2025-04-01T20:11:01Z

Closes #55
Closes #56

~~The idea is that we expose the MCP server and the Express app that can then be extended with other endpoints - so it can be used in the platform MCP server.~~

The new proposed solution is to expose MCP server and tools/prompt/resources functionalty as additional library and keep the stdio (and the Actor for now - legacy).

library entrypoint

MQ37 · 2025-04-03T07:35:55Z

I splitted the tools to separate file in tools dir. Also changed types for two basic tools ActorTool - calls an Actor; and Internal Tool has call function and does some logic which returns MCP response. Also prepared dir toolkits where we can create basic list of tools like Actor discovery toolkit or auto loading Actor toolkit, ...

jirispilka · 2025-04-04T19:44:02Z

@MQ37 thanks! But it would be better to have separate PRs, it is becoming very big and intractable.

First, we need to decouple Actor and server. You also need to separate README, Actor README is going to be different from the package README

MQ37 · 2025-04-05T11:20:08Z

@MQ37 thanks! But it would be better to have separate PRs, it is becoming very big and intractable.

First, we need to decouple Actor and server. You also need to separate README, Actor README is going to be different from the package README

I agree with you that this is a large PR but I think these changes go hand in hand with each other and it would be easier to do both of them at once for us - I am also working on integrating this into the apify-mcp-server repo.

jirispilka · 2025-04-05T19:51:50Z

I'm afraid, I don't really like it, there is no clear separation of the Actor code and a new mcp-server code :(
That was the requirement of the issue.
Instead, there is a mix of tools.ts, toolkits, tools directory, which I don't quite get.
I would first finish one thing, settle the structure and then introduce a new stuff

MQ37 · 2025-04-06T08:04:58Z

I'm afraid, I don't really like it, there is no clear separation of the Actor code and a new mcp-server code :( That was the requirement of the issue. Instead, there is a mix of tools.ts, toolkits, tools directory, which I don't quite get. I would first finish one thing, settle the structure and then introduce a new stuff

It is still a WIP, but the main idea is that all tools will be in the tools/ directory (same for resources/ and prompts/ as suggested in #56) and the server entry points and MCP server class are separated (so we can easily use this in the apify-mcp-server based on #55). So the main.ts is the entry point for the Actor and stdio.ts is the entry point for the stdio version of the server. There are some places that still need work and some moving around. What other project structure do you suggest then?

… up old tools.ts, separate Actor readme

… Actor use case

…unction and updating types.

…ly export ApifyMcpServer from the package

jirispilka · 2025-04-09T20:44:55Z

I think the structure is better now .... but this PR is huge because we couple together structure and tools refactoring.
We will need to live with that and test properly.

There are two things I'm not happy with:

I don't like that we are passing apifyMcpServer and mcpServer to a tool
We need to handle errors better by sending server.sendLoggingMessage ( )

So this is still work in progress

MQ37

Just a few comments, otherwise nice work @jirispilka 👍

MQ37 · 2025-04-11T07:33:57Z

src/actor/const.ts

+    actors: [
+        'apify/instagram-scraper',
+        'apify/rag-web-browser',
+        'lukaskrivka/google-maps-with-contact-details',
+    ],


The default list of loaded Actors/tools is not only Actor related (is used in stdio.ts), I think we should move it to src/const.ts

src/main.ts

src/tools/actor.ts

* docs: document input schema processing (#58) document Actor input schema processing in README * add VS Code instructions to README (#62) * Add vs code instructions --------- Co-authored-by: mbaiza27 <[email protected]> Co-authored-by: Jiří Spilka <[email protected]> * ci: run prerelase manually (#70) run prerelase manually * get apify api url base from env var --------- Co-authored-by: Marc Baiza <[email protected]> Co-authored-by: mbaiza27 <[email protected]> Co-authored-by: Jiří Spilka <[email protected]>

) * fix: add http request/response, fix lint * feat: add client for /mcp endpoint * fix: fork modelcontextprotocol repository * fix: refactor server.ts

* docs: document input schema processing (#58) document Actor input schema processing in README * add VS Code instructions to README (#62) * Add vs code instructions --------- Co-authored-by: mbaiza27 <[email protected]> Co-authored-by: Jiří Spilka <[email protected]> * initial working draft of proxy actorized mcp servers * fix tests * fix proxy tool name handling * ci: run prerelase manually (#70) run prerelase manually * organize, fix passing of apify token * fix env var name * fix imports * get standby url from actor id, check if is mcp server based on actor name for now * Actor MCP server load default Actors * get standbyUrlBase from env var * fix standby url base, use dns friendly owner name * get mcp path from definition * Remove unused import from actor.ts * console.log to logger * Remove extra slash from MCP server URL construction * Simplify actor tools loading logic * Refactor actorOwnerDNSFriendly to handle special characters * Add TODO comment for reworking actor definition fetch logic * refactor mcp actor and utils, use real Actor ID as standby URL * fix: get-actor-definition-default-build (#73) * fix get default build in get actor definition * fix tests * fix double import, lint --------- Co-authored-by: Marc Baiza <[email protected]> Co-authored-by: mbaiza27 <[email protected]> Co-authored-by: Jiří Spilka <[email protected]> Co-authored-by: Jiri Spilka <[email protected]>

jirispilka · 2025-04-16T14:50:51Z

I know, one test is failing x I didn't have time to do it before leaving the office.

MQ37 · 2025-04-16T19:55:50Z

@jirispilka thank you for fixing the tests 👍 Tested Actorized MCPs and normal Actors in Claude desktop (stdio) and on staging via SSE and everything works ✔️

jirispilka · 2025-04-16T20:26:55Z

@MQ37 cool. Thanks! We should also test existing Actor, whether it works via SSE. I'll create a task for this

MQ37 · 2025-04-17T07:58:56Z

@MQ37 cool. Thanks! We should also test existing Actor, whether it works via SSE. I'll create a task for this

Tested again as Actor and it works 👍

MQ37 added 5 commits April 1, 2025 22:10

wip snapshot, decouple logic, split stdio cli, http server and prepare

bb1c10b

library entrypoint

lint

03d772c

improve

0ccc995

expose lib

4c1e3df

split tools, add call for internal tools, prepare toolkits

255b91d

MQ37 added 2 commits April 3, 2025 12:35

Move tool exports to toolkit modules

ffc05ce

Refactor tool exports and update library entry points

9241f03

MQ37 and others added 15 commits April 7, 2025 10:31

remove toolkits (move to tools index), pass apify token as arg, clean…

59a9e0a

… up old tools.ts, separate Actor readme

fix stdio.js path, get Apify token from env var for stdio and standby…

99bee44

… Actor use case

export helper functions, fix typos and minor issues

e0af0b1

tsconfig remove root dir

0697fc2

Refactor actor run data handling by moving logic to getActorRunData f…

e982411

…unction and updating types.

split actors.ts

ca13d54

fix test imports

2c7edaf

simplify mcp server class, move to actors/call.ts

f8ad397

decouple processParamsAndUpdateTools

204b358

dataset push only in normal mode for debugging

8ed82ab

add server tests

915faf8

update processParamsAndUpdateTools docs

f1d630d

fix: clearly separate Actor related code and rest of the codebase. On…

ce711e9

…ly export ApifyMcpServer from the package

fix: Update packages, add TODOs

b78fb6b

fix: lint issue

8735e8f

fix: rename server to ActorsMcpServer

6879923

MQ37 commented Apr 11, 2025

View reviewed changes

MQ37 force-pushed the feat/decouple branch 3 times, most recently from 0325f4c to 52ccf0c Compare April 11, 2025 13:01

bump beta version

b0b558e

MQ37 force-pushed the feat/decouple branch from a02b8ab to b0b558e Compare April 11, 2025 13:03

Apify Release Bot and others added 10 commits April 11, 2025 13:04

chore(release): Update changelog and package version [skip ci]

d8e7294

move defaults from actors/ to consts in src/

0a836c9

chore(release): Update changelog and package version [skip ci]

a9d3b65

feat: add new specification regarding json response streamable http (#72

e918d26

) * fix: add http request/response, fix lint * feat: add client for /mcp endpoint * fix: fork modelcontextprotocol repository * fix: refactor server.ts

chore(release): Update changelog and package version [skip ci]

95c311d

fix: lint issues, disable feat/decouple

d8514f9

Merge remote-tracking branch 'origin/feat/decouple' into feat/decouple

b538c04

fix: merge master

b27a607

fix: unit-tests

118f049

jirispilka marked this pull request as ready for review April 16, 2025 20:27

jirispilka self-requested a review April 16, 2025 20:27

jirispilka approved these changes Apr 16, 2025

View reviewed changes

jirispilka changed the title ~~feat: decouple logic~~ feat: Decouple Actor from mcp-server, add ability to call Actorized MCP and load tools Apr 16, 2025

MQ37 and others added 4 commits April 16, 2025 23:30

fix stream not readable and apify api url base for Actor server

76eba1c

fix: logic when starting server

f5310a6

fix: server.ts

aadd4a0

fix: change port to not collide with inspector

5d0a84a

jirispilka merged commit fe8d9c2 into master Apr 17, 2025
2 checks passed

jirispilka deleted the feat/decouple branch April 17, 2025 09:18

feat: Decouple Actor from mcp-server, add ability to call Actorized MCP and load tools #59

feat: Decouple Actor from mcp-server, add ability to call Actorized MCP and load tools #59

Uh oh!

Conversation

MQ37 commented Apr 1, 2025 • edited by jirispilka Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MQ37 commented Apr 3, 2025

Uh oh!

jirispilka commented Apr 4, 2025

Uh oh!

MQ37 commented Apr 5, 2025

Uh oh!

jirispilka commented Apr 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MQ37 commented Apr 6, 2025

Uh oh!

jirispilka commented Apr 9, 2025

Uh oh!

MQ37 left a comment

Choose a reason for hiding this comment

Uh oh!

MQ37 Apr 11, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jirispilka commented Apr 16, 2025

Uh oh!

MQ37 commented Apr 16, 2025

Uh oh!

jirispilka commented Apr 16, 2025

Uh oh!

MQ37 commented Apr 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

MQ37 commented Apr 1, 2025 •

edited by jirispilka

Loading

jirispilka commented Apr 5, 2025 •

edited

Loading

MQ37 commented Apr 17, 2025 •

edited

Loading