-
Notifications
You must be signed in to change notification settings - Fork 2
Add playground guide #214
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Add playground guide #214
Changes from all commits
Commits
Show all changes
20 commits
Select commit
Hold shift + click to select a range
46517db
Add playground guide
samuv 8c8b450
fix: typo
samuv aac832b
fix: typo
samuv 56f7124
fix: adjust sidebar position, remove some images, and decapitalize pl…
samuv bfc34fd
3 spaces on list
samuv 59afb20
3 spaces list
samuv 178f3f3
extra explanation before second example
samuv 41ad1c5
remove "feature"
samuv 305334b
starts instead of spins up
samuv c92e7a3
rewording validation phrase
samuv 2942bbf
learn more
samuv ca4ac33
Browse instead of Check out
samuv cb72b01
Recommended instead of Best
samuv 4ad9337
rewording the description
samuv a14b25e
rewording the goal
samuv 300e400
respond
samuv b420768
rewording thv mcp description
samuv 9e72964
fix: format
samuv 630bf4f
feat: add playgroun on next step for run server and other adjustments
samuv 5a0ae0d
refactor: change title for audit logs
samuv File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,239 @@ | ||
| --- | ||
| title: Test MCP servers with playground | ||
| description: | ||
| Use the playground feature to test and validate MCP servers directly in the | ||
| ToolHive UI with AI model providers. | ||
| --- | ||
|
|
||
| import useBaseUrl from '@docusaurus/useBaseUrl'; | ||
| import ThemedImage from '@theme/ThemedImage'; | ||
|
|
||
| ToolHive's playground lets you test and validate MCP servers directly in the UI | ||
| without requiring additional client setup. This streamlined testing environment | ||
| helps you quickly evaluate functionality and behavior before deploying MCP | ||
| servers to production environments. | ||
|
|
||
| ## Key capabilities | ||
|
|
||
| ### Instant testing of MCP servers | ||
|
|
||
| Enter an AI model API key, select your MCP servers and tools, and begin testing | ||
| immediately in the desktop app. The playground eliminates the friction of | ||
| setting up external AI clients just to validate that your MCP servers work | ||
| correctly. | ||
|
|
||
| ### Detailed interaction logs | ||
|
|
||
| See tool details, parameters, and execution results directly in the UI, ensuring | ||
| full visibility into tool performance and responses. Every interaction is | ||
| logged, making it easy to understand exactly what your MCP servers are doing and | ||
| how they respond to requests. | ||
|
|
||
| ### Integrated ToolHive management | ||
|
|
||
| The playground includes a built-in MCP server that lets you manage your other | ||
| MCP servers directly through natural language commands. You can list servers, | ||
| check their status, start or stop them, and perform other management tasks using | ||
| conversational AI. | ||
|
|
||
| ## Getting started | ||
|
|
||
| To start using the playground: | ||
|
|
||
| 1. **Access the playground**: Click the **Playground** tab in the ToolHive UI | ||
| navigation bar. | ||
|
|
||
| 2. **Configure API keys**: Click **Configure your API Keys** to set up access to | ||
| AI model providers: | ||
| - **OpenAI**: Enter your OpenAI API key to use GPT models | ||
| - **Anthropic**: Add your Anthropic API key for Claude models | ||
| - **Google**: Configure Google AI API key for Gemini models | ||
| - **xAI**: Set up xAI API key for Grok models | ||
| - **OpenRouter**: Add OpenRouter API key for access to multiple model | ||
| providers | ||
|
|
||
| 3. **Select MCP tools**: Click the tools icon to manage which MCP servers and | ||
| tools are available in the playground. | ||
|
|
||
| <ThemedImage | ||
| alt='ToolHive playground tools management showing available MCP tools' | ||
| sources={{ | ||
| light: useBaseUrl( | ||
| '/img/toolhive/toolhive-ui-playground-thv-mcp-light.webp' | ||
| ), | ||
| dark: useBaseUrl('/img/toolhive/toolhive-ui-playground-thv-mcp-dark.webp'), | ||
| }} | ||
| title='MCP tools management interface' | ||
| /> | ||
|
|
||
| - View all your running MCP servers | ||
| - Enable or disable specific tools from each server | ||
| - Search and filter tools by name or functionality | ||
| - The `toolhive mcp` server is included by default, providing management | ||
| capabilities | ||
|
|
||
| 4. **Start testing**: Begin chatting with your chosen AI model. The model will | ||
| have access to all enabled MCP tools and can execute them based on your | ||
| requests. | ||
|
|
||
| ## Using the playground | ||
|
|
||
| ### Testing MCP server functionality | ||
|
|
||
| Use the playground to validate that your MCP servers work as expected: | ||
|
|
||
| ```text | ||
| Can you list all my MCP servers and show their current status? | ||
| ``` | ||
|
|
||
| The AI will use the `list_servers` tool from the ToolHive MCP server to provide | ||
| a comprehensive overview of your server status. | ||
|
|
||
| <ThemedImage | ||
| alt='ToolHive playground showing AI response with MCP tool execution results' | ||
| sources={{ | ||
| light: useBaseUrl( | ||
| '/img/toolhive/toolhive-ui-playground-chat-response-light.webp' | ||
| ), | ||
| dark: useBaseUrl( | ||
| '/img/toolhive/toolhive-ui-playground-chat-response-dark.webp' | ||
| ), | ||
| }} | ||
| title='Chat response showing tool execution and results' | ||
| /> | ||
|
|
||
| Or test that an individual MCP tool is working as expected: | ||
|
|
||
| ```text | ||
| Use the GitHub MCP server to search for recent issues in the microsoft/vscode repository | ||
| ``` | ||
|
|
||
| If you have the GitHub MCP server running, the AI will execute the appropriate | ||
| GitHub API calls and return formatted results. | ||
|
|
||
| ### Managing servers through conversation | ||
|
|
||
| The ToolHive desktop app automatically starts a dedicated MCP server | ||
| (`toolhive mcp`) that orchestrates ToolHive operations through natural language | ||
| commands. This approach provides several key benefits: | ||
|
|
||
| - **Unified interface**: Manage your MCP infrastructure using the same | ||
| conversational AI interface you use for testing | ||
| - **Contextual operations**: The AI understands your current server state and | ||
| can make intelligent decisions about which servers to start, stop, or | ||
| troubleshoot | ||
| - **Reduced complexity**: No need to switch between the chat interface and | ||
| traditional UI controls—everything can be done through conversation | ||
| - **Audit trail**: All management operations are logged in the same transparent | ||
| way as tool executions, providing clear visibility into what actions were | ||
| taken | ||
|
|
||
| Take advantage of these integrated ToolHive management tools: | ||
|
|
||
| ```text | ||
| Start the fetch MCP server for me | ||
| ``` | ||
|
|
||
| ```text | ||
| Stop all unhealthy MCP servers | ||
| ``` | ||
|
|
||
| ```text | ||
| Show me the logs for the meta-mcp server | ||
| ``` | ||
|
|
||
| ### Validating tool responses | ||
|
|
||
| The playground shows detailed information about each tool execution: | ||
|
|
||
| - **Tool name and description**: What tool was called and its purpose | ||
| - **Input parameters**: The exact parameters passed to the tool | ||
| - **Execution status**: Whether the tool succeeded or failed | ||
| - **Response data**: The complete response from the tool | ||
| - **Timing information**: How long the tool took to execute | ||
|
|
||
| This visibility helps you understand exactly how your MCP servers are behaving | ||
| and identify any issues with tool implementation or configuration. | ||
|
|
||
| ## Recommended practices | ||
|
|
||
| ### API key security | ||
|
|
||
| - Use dedicated API keys for testing that have appropriate rate limits | ||
| - Regularly rotate API keys used in development environments | ||
| - Consider using API keys with restricted permissions for testing purposes | ||
|
|
||
| ### Server management | ||
|
|
||
| - Start only the MCP servers you need for testing to improve performance | ||
| - Use the playground to validate new server configurations before connecting | ||
| them to production AI clients | ||
| - Test different combinations of tools to understand how they work together | ||
|
|
||
| ### Testing workflow | ||
|
|
||
| 1. **Isolated testing**: Test individual MCP servers one at a time to validate | ||
| their functionality | ||
| 2. **Integration testing**: Enable multiple servers to test how they work | ||
| together | ||
| 3. **Performance validation**: Monitor tool execution times and responses under | ||
| different loads | ||
| 4. **Error handling**: Intentionally trigger error conditions to validate proper | ||
| error handling | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| ### Common issues | ||
|
|
||
| <details> | ||
| <summary>API key not working</summary> | ||
|
|
||
| If your API key isn't working: | ||
|
|
||
| 1. Verify the API key is correct and has proper permissions | ||
| 2. Check that you have sufficient quota or credits with the provider | ||
| 3. Ensure the API key hasn't expired or been revoked | ||
| 4. Try creating a new API key from the provider's dashboard | ||
|
|
||
| </details> | ||
|
|
||
| <details> | ||
| <summary>MCP tools not appearing</summary> | ||
|
|
||
| If your MCP server tools aren't showing up: | ||
|
|
||
| 1. Verify the MCP server is running (check the **MCP Servers** page) | ||
| 2. Click the tools icon and ensure the server's tools are enabled | ||
| 3. Try restarting the MCP server if it shows as unhealthy | ||
| 4. Check the server logs for any error messages | ||
|
|
||
| </details> | ||
|
|
||
| <details> | ||
| <summary>Tool execution failing</summary> | ||
|
|
||
| If tools are failing to execute: | ||
|
|
||
| 1. Check the tool's parameter requirements in the audit log | ||
| 2. Verify any required secrets or environment variables are configured | ||
| 3. Ensure the MCP server has necessary permissions (network access, file system | ||
| access) | ||
| 4. Review the server logs for detailed error information | ||
|
|
||
| </details> | ||
|
|
||
| ## Next steps | ||
|
|
||
| - Learn about [client configuration](./client-configuration.mdx) to connect | ||
| ToolHive to external AI applications | ||
| - Set up [secrets management](./secrets-management.md) for secure handling of | ||
| API keys and tokens | ||
| - Explore [network isolation](./network-isolation.mdx) for enhanced security | ||
| when testing untrusted MCP servers | ||
| - Browse the [registry](./registry.mdx) to discover new MCP servers to test in | ||
| the playground | ||
|
|
||
| ## Related information | ||
|
|
||
| - [Run MCP servers](./run-mcp-servers.mdx) | ||
| - [Install ToolHive](./install.mdx) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.