Add MongoDB Assistant Tools #472
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR introduces MongoDB Assistant tools for the MCP server, enabling natural language search and knowledge source discovery. It adds integration with the MongoDB Assistant AI chatbot and search service.
- Scaffolds a new
AssistantToolBaseclass for MongoDB Assistant tool interactions - Implements two initial tools:
list_knowledge_sourcesandsearch_knowledge - Adds comprehensive test coverage with mocked API responses
Reviewed Changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/common/config.ts | Adds assistantBaseUrl configuration option |
| src/tools/tool.ts | Extends ToolCategory type to include "assistant" |
| src/tools/assistant/assistantTool.ts | Base class for Assistant tools with common functionality |
| src/tools/assistant/list_knowledge_sources.ts | Tool for listing available knowledge sources |
| src/tools/assistant/search_knowledge.ts | Tool for searching the knowledge base |
| src/tools/assistant/tools.ts | Exports array of Assistant tools |
| src/server.ts | Registers Assistant tools with the server |
| tests/integration/helpers.ts | Updates response element types to support metadata |
| tests/integration/tools/assistant/assistantHelpers.ts | Test utilities for Assistant tool testing |
| tests/integration/tools/assistant/listKnowledgeSources.test.ts | Integration tests for list knowledge sources tool |
| tests/integration/tools/assistant/searchKnowledge.test.ts | Integration tests for search knowledge tool |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| import { LogId } from "../../common/logger.js"; | ||
|
|
||
| export const SearchKnowledgeToolArgs = { | ||
| query: z.string().describe("A natural language query to search for in the knowledge base"), |
There was a problem hiding this comment.
nit: can we add a bit more detail here? this is what the LLM will use to understand that you want it to make a generic natural question that the user haave.
| this.requiredHeaders = new Headers({ | ||
| "x-request-origin": "mongodb-mcp-server", | ||
| "user-agent": serverVersion ? `mongodb-mcp-server/v${serverVersion}` : "mongodb-mcp-server", | ||
| }); |
There was a problem hiding this comment.
[q] why does this need to be initialized here at the tool level? can we also move this logic into the api layer?
also, we should probably use the userAgent from the apiClient and make sure we use the same name everywhere
There was a problem hiding this comment.
why does this need to be initialized here at the tool level? can we also move this logic into the api layer?
The "API layer" as it exists seems to be tailored to Atlas - this is a totally separate server/API. If we prefer I could create an equivalent assistant/apiClient.ts file but personally that seems unnecessary at this point.
we should probably use the userAgent from the apiClient
- Our server firewall allowlists specific user agents and right now that's configured to be
mongodb-mcp-server - nit: The
apiClientusesUser-Agent: AtlasMCPwhich again seems to refer to Atlas specifically
| /** | ||
| * Mocks fetch for assistant API calls | ||
| */ | ||
| interface MockedAssistantAPI { |
There was a problem hiding this comment.
nit: if we're mocking the api calls i'd rather call these unit tests, could you please move the tests to the unit folder and the helpers?
if you have any staging / cloud-dev environment, we could configure integration tests as well, let me know on slack if you do as I can help setting up a GH variable for the staging environment, otherwise we could keep it with unit tests only which is fine
There was a problem hiding this comment.
Gotcha that's totally reasonable. I'll move these mocked tests to tests/unit. It's a bit awkward because I think I still want the integration test flow (i.e. mcpClient.callTool({ ... })) but I'd like to use the mocks to control the output to confirm that everything is formatted/handled appropriately.
For integration tests, we could try hitting https://knowledge-dev.mongodb.com. What GH variables do we need to support this?
There was a problem hiding this comment.
I created an env variable called TEST_ASSISTANT_BASE_URL in GH actions, you can make it so your integration tests get configured to that if the env is set.
here's an atlas example. you'd need to add the logic for using it as well
- name: Run tests
env:
MDB_MCP_API_CLIENT_ID: ${{ secrets.TEST_ATLAS_CLIENT_ID }}
MDB_MCP_API_CLIENT_SECRET: ${{ secrets.TEST_ATLAS_CLIENT_SECRET }}
MDB_MCP_API_BASE_URL: ${{ vars.TEST_ATLAS_BASE_URL }}
run: npm test -- tests/integration/tools/atlas
|
The only thing missing from my side in this PR is to add some accuracy tests, to make sure that the models we test with do know when to call this tool. In the tests/accuracy folder there are a few examples that can be used as inspiration. The idea is to write a few prompts that we expect to trigger our tool and ensure that the tool is called with the specific parameters. |
|
Good to see there is work being done on this area. As I mentioned on the issue I created, I have always found myself going through the documentation, but also making sure that what I'm reading is available on the version that I have installed. It is not always the case that people are on the latest of the latest, so would there be a way to "hint" the MCP about the version you need the documentation to be? |
|
|
||
| export class ListKnowledgeSourcesTool extends AssistantToolBase { | ||
| public name = ListKnowledgeSourcesToolName; | ||
| protected description = "List available data sources in the MongoDB Assistant knowledge base"; |
There was a problem hiding this comment.
When testing locally I was able to trigger it explicitly by "listing knowledge topics.
After extending description a little bit I was able to call this tool more accurately.
"list mongodb topics for me"
| protected description = "List available data sources in the MongoDB Assistant knowledge base"; | |
| protected description = "List available data sources in the MongoDB Assistant knowledge base. Covers t knowledge topics for MongoDB database"; |
|
This PR has gone 30 days without any activity and meets the project's definition of "stale". This will be auto-closed if there is no new activity over the next 30 days. If the issue is still relevant and active, you can simply comment with a "bump" to keep it open, or add the label "not_stale". Thanks for keeping our repository healthy! |
|
bump |
|
@nlarew if you have a chance to update the PR, it'd be great to get this in. Sorry for letting this get stale. |
8 participants
Proposed changes
(EAI-1266) Knowledge Search on MCP
Adds a new class of
Assistanttools for interacting with the MongoDB Assistant (AI Chatbot + Search).This PR scaffolds the generic
AssistantToolBaseclass as well as two initial tools:list_knowledge_sources- pulls a list of all available data sources. Useful for knowing what data is available and for filtering thesearch_knowledgesearch_knowledge- runs a natural language search against the MongoDB Assistant knowledge base. Returns content chunks for the most relevant results.Checklist