Skip to content

test: refactor integration test framework for full MCP validation#2711

Open
anubhav756 wants to merge 6 commits intomainfrom
anubhav-framework
Open

test: refactor integration test framework for full MCP validation#2711
anubhav756 wants to merge 6 commits intomainfrom
anubhav-framework

Conversation

@anubhav756
Copy link
Contributor

@anubhav756 anubhav756 commented Mar 11, 2026

Overview

This PR refactors the underlying test methodology across MCP Toolbox to natively support integration testing using MCP JSON-RPC requests.

Changes

  • Introduced new native MCP test helpers (RunMCPToolInvokeTest, RunMCPExecuteSqlToolInvokeTest, RunMCPToolInvokeSimpleTest) in tests/tool.go.
  • Added standard JSON-RPC request structures (jsonrpc.JSONRPCRequest) to simulate robust MCP payloads.
  • Rerouted testing endpoints away from the /api HTTP implementation straight to the core /mcp pathway.

Note

This PR is Part 1 of 5 to fully migrate MCP Toolbox onto MCP endpoints and remove the legacy /api REST endpoints.

@anubhav756 anubhav756 self-assigned this Mar 11, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Mar 11, 2026

Link Resolution Note

Local links and directory changes work differently on GitHub than on the docsite. You must ensure fixes pass the GitHub check and also work with hugo server.
See Link Checking and Fixing with Lychee for more details.

Summary

Status Count
🔍 Total 44
✅ Successful 38
⏳ Timeouts 0
👻 Excluded 0
❓ Unknown 0
🚫 Errors 1
⛔ Unsupported 0

Errors per input

Errors in DEVELOPER.md

  • [ERROR] https://play.dgraph.io/ | Network error: Connection failed. Check network connectivity and firewall settings (error sending request for url (https://play.dgraph.io/)): Connection failed. Check network connectivity and firewall settings

Full Github Actions output

@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on enhancing the integration test framework to provide more robust and specific validation for Multi-Cloud Platform (MCP) tool invocation endpoints. The changes involve a significant refactoring of test configurations and functions, moving towards a standardized approach for testing MCP tool interactions. By centralizing error handling and introducing dedicated test helpers, the PR aims to improve the clarity, efficiency, and maintainability of the test suite for MCP components.

Highlights

  • Refactored Integration Test Framework: The integration test framework has been significantly refactored to specifically validate Multi-Cloud Platform (MCP) tool invocation endpoints, enhancing clarity and maintainability.
  • Centralized Error Handling: Error response structures and helper functions were moved to internal/server/mcp.go, centralizing error handling logic for MCP-related API endpoints.
  • New MCP Test Helper Functions: Several new test helper functions (RunMCPToolInvokeSimpleTest, RunMCPToolInvokeParametersTest, RunMCPToolInvokeTest, RunMCPToolInvokeWithTemplateParameters) were introduced in tests/tool.go to standardize and streamline MCP tool invocation testing.
  • Simplified HTTP Integration Tests: The tests/http/http_integration_test.go file was simplified by replacing verbose custom test logic with calls to the new, more generic RunMCPToolInvokeParametersTest helper.
  • Improved Local Testing Setup: Default values for ClientId and ServiceAccountEmail are now set in tests/auth.go if environment variables are not present, improving the local testing experience.
Changelog
  • DEVELOPER.md
    • Updated documentation to reference the new /mcp tools/call endpoint test.
  • internal/server/api.go
    • Removed errResponse struct and newErrResponse helper function.
  • internal/server/mcp.go
    • Moved errResponse struct and newErrResponse helper function from api.go.
  • internal/server/mcp_test.go
    • Imported additional packages including chi/v5, prompts, tools, and parameters.
  • tests/auth.go
    • Initialized ClientId and ServiceAccountEmail with default values if environment variables are empty.
  • tests/http/http_integration_test.go
    • Removed bytes import and added os import.
    • Refactored TestHttpToolEndpoints to use tests.RunMCPToolInvokeTest.
    • Rewrote runQueryParamInvokeTest and runAdvancedHTTPInvokeTest to use tests.RunMCPToolInvokeParametersTest.
  • tests/option.go
    • Renamed InvokeTestConfig to MCPToolInvokeTestConfig and InvokeTestOption to MCPToolInvokeTestOption.
    • Updated comments and function signatures to reflect the new MCPToolInvokeTest naming convention.
    • Updated comments for ExecuteSqlTestConfig and TemplateParameterTestConfig to reference RunMCPExecuteSqlToolInvokeTest and RunMCPToolInvokeWithTemplateParameters respectively.
  • tests/tool.go
    • Renamed RunToolInvokeTest to RunMCPToolInvokeTest and updated its internal configuration type.
    • Added new test helper functions: RunMCPToolInvokeSimpleTest, RunMCPToolInvokeParametersTest, RunMCPToolInvokeTest, and RunMCPToolInvokeWithTemplateParameters.
    • Updated RunMCPExecuteSqlToolInvokeTest to use MCPToolInvokeTestOption.
Activity
  • No human activity has been recorded on this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

The pull request migrates tool invocation tests from a direct HTTP API to a new JSON-RPC 2.0 based /mcp endpoint, standardizing error handling and test setup. Documentation in DEVELOPER.md was updated to reflect the new RunMCPToolInvokeTest for tool execution. New helper functions and mock data were introduced in internal/server/mcp_test.go to support the JSON-RPC format, and tests/auth.go now sets default client IDs for test reliability. Existing HTTP integration tests and various tool invocation test functions in tests/tool.go and tests/option.go were refactored to use the new MCP endpoint and JSON-RPC request structure. Review comments highlight that several tests still use the old request body format for the /mcp endpoint, which will cause failures, and point out a typo in requiresClientAuthrorization in mcp_test.go. Additionally, a resource management improvement is suggested to move defer resp.Body.Close() immediately after http.DefaultClient.Do(req) in runRequest for robustness.

I am having trouble creating individual review comments. Click here to see my feedback.

tests/tool.go (1447-1448)

critical

This test and several others below it (e.g., RunPostgresListViewsTest, RunPostgresListSchemasTest) have been updated to use the /mcp endpoint, but the requestBody is still in the old format. The /mcp endpoint expects a JSON-RPC 2.0 request, so these tests will fail.

The request body needs to be wrapped in a proper JSON-RPC structure. For example:

{
  "jsonrpc": "2.0",
  "id": "some-id",
  "method": "tools/call",
  "params": {
    "name": "list_tables",
    "arguments": {
      "table_names": ""
    }
  }
}

DEVELOPER.md (184)

medium

This line seems to be a leftover from the previous description and is now inaccurate for RunMCPToolInvokeTest. This test executes a tool, it does not return the tool's manifest.

internal/server/mcp_test.go (1231)

medium

There is a typo in the field name requiresClientAuthrorization. It should be requiresClientAuthorization.

requiresClientAuthorization: true,

internal/server/mcp_test.go (1365-1374)

medium

The defer resp.Body.Close() should be placed immediately after the successful http.DefaultClient.Do(req) call. This ensures that the response body is closed even if subsequent operations like io.ReadAll fail or panic. It's a standard Go idiom for resource management.

resp, err := http.DefaultClient.Do(req)
if err != nil {
	return nil, nil, fmt.Errorf("unable to send request: %w", err)
}
defer resp.Body.Close()

respBody, err := io.ReadAll(resp.Body)
if err != nil {
	return nil, nil, fmt.Errorf("unable to read request body: %w", err)
}

@anubhav756 anubhav756 force-pushed the anubhav-framework branch 3 times, most recently from ea98ec8 to 3240483 Compare March 11, 2026 15:55
@anubhav756 anubhav756 changed the title test: refactor integration test framework for full MCP validation test: refactor integration test framework to use MCP JSON-RPC Mar 11, 2026
@anubhav756 anubhav756 marked this pull request as ready for review March 11, 2026 19:29
@anubhav756 anubhav756 requested a review from a team as a code owner March 11, 2026 19:29
@anubhav756 anubhav756 added the tests: run Label to trigger Github Action tests. label Mar 11, 2026
@anubhav756 anubhav756 force-pushed the anubhav-framework branch 2 times, most recently from 2ef806a to d3ce2ce Compare March 12, 2026 10:54
@anubhav756 anubhav756 changed the title test: refactor integration test framework to use MCP JSON-RPC test: refactor integration test framework for full MCP validation Mar 12, 2026
@anubhav756 anubhav756 force-pushed the anubhav-framework branch 4 times, most recently from 9ebf274 to 64bffaa Compare March 16, 2026 10:24
Copy link
Contributor

@Yuan325 Yuan325 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we want to rename everything with a MCP? Since the entire Toolbox is being updated to MCP anyway, I'm wondering if keeping the existing names might keep things cleaner. I'm slightly concerned that adding 'MCP' to everything might end up being a bit redundant~

}

// RunMCPToolInvoke runs the tool invoke endpoint
func RunMCPToolInvokeTest(t *testing.T, select1Want string, options ...MCPToolInvokeTestOption) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this specific towards the http source?

@anubhav756 anubhav756 force-pushed the anubhav-framework branch 2 times, most recently from e842e1c to 63d049f Compare March 17, 2026 12:00
@anubhav756
Copy link
Contributor Author

/gcbrun

1 similar comment
@anubhav756
Copy link
Contributor Author

/gcbrun

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tests: run Label to trigger Github Action tests.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants