Skip to content

Conversation

@dmartinol
Copy link
Collaborator

Changes to address #5 and other structural updates.

Python Version & Dependencies

  • Updated to Python 3.10+ support
  • Updated to use mcp package instead of older dependencies

MCP Specification Compliance

  • Updated to comply with MCP spec 2025-06-18

CLI Interface Review

  • Added -- separator for command arguments (stdio transport)
  • Added --debug option for detailed execution tracking
  • Added OAuth options: --client-id, --client-secret, --auth-token

Transport System Complete Rewrite

  • Added SSE and Streamable HTTP transports (--transport sse, --transport http) with CLI support
  • Enhanced HTTP transport with full OAuth 2.0 flows
  • OAuth 2.0 Dynamic Client Registration (RFC 7591)
  • Pre-registered OAuth client support with automatic browser opening for OAuth authentication
  • Bearer token authentication for both HTTP and SSE

Enhanced Error Handling & Debugging

  • Verbose logging with --verbose and --debug flags

Documentation Updates

  • Complete README rewrite with all transport examples
  • OAuth authentication guide with multiple flows
  • CLI options reference (11 → 20+ options)
  • Advanced debugging examples

dmartinol and others added 11 commits October 3, 2025 09:36
- Add MCPTransport abstract base class with standardized interface
- Refactor JSONRPCTransport to StdioTransport implementing MCPTransport
- Update ValidationContext to use MCPTransport interface
- Add new fields for HTTP transport support (endpoint, transport_type)
- Maintain backward compatibility via JSONRPCTransport alias
- Add proper transport cleanup with close() method

This establishes the foundation for supporting multiple transport
types including the planned HTTP and SSE transports while keeping
all existing functionality intact.

🤖 Generated with [Claude Code](https://claude.ai/code)

Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
- Introduce HTTP transport implementation for MCP communication, allowing for HTTP-based interactions.
- Update argument parser to include transport options and validate endpoint URLs.
- Refactor validation orchestrator to support both stdio and HTTP transports, ensuring proper cleanup and initialization.
- Add transport factory for creating transport instances based on type, enhancing modularity and maintainability.

This update lays the groundwork for more flexible communication methods in MCP validation, improving usability and integration capabilities.

Signed-off-by: D. Martinoli <dmartino@redhat.com>
Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
- use mcp package
- linting code
- auth options

Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
- exposed tools to security check

Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
- Add complete HTTP transport implementation using MCP SDK's streamablehttp_client
- Support three authentication strategies:
  * OAuth 2.0 Dynamic Client Registration (RFC 7591) - automatic
  * Pre-registered OAuth applications - manual setup
  * Personal access tokens - direct authentication
- Implement OAuth callback server with automatic browser opening
- Add pre-flight authentication checks to avoid SDK crashes
- Update CLI to display authentication method being used
- Enable context sharing between validators for accurate tool counts
- Add comprehensive OAuth documentation to README

Successfully tested with GitLab's MCP endpoint discovering 10 tools:
get_pipeline_jobs, get_issue, create_issue, create_merge_request,
get_merge_request, and 5 more GitLab MCP tools.

🤖 Generated with [Claude Code](https://claude.ai/code)

Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
@caxu-rh
Copy link

caxu-rh commented Oct 8, 2025

While running against the GitLab MCP server:

$ uv run mcp-validate --debug --verbose --transport http --endpoint http://localhost:8080/api/v4/mcp
[...]
🔍 ✅ Received OAuth callback with authorization code
🔍 🔄 OAuth callback server stopped
🔍 ✅ HTTP transport and MCP session initialized successfully
🔍 ✅ Transport initialized successfully
[10:48:00.538] [EXEC-INFO] 🔄 HTTP transport initialized: Endpoint: http://localhost:8080/api/v4/mcp
[10:48:00.538] [EXEC-INFO] 🔄 Setting up validation context
🔍 Loading validation profile: comprehensive
[10:48:00.538] [EXEC-INFO] 🔄 Creating validators: Profile: comprehensive
[10:48:00.538] [REGISTRY-INFO] Initializing RegistryValidator with config: {'enabled': True, 'required': True, 'timeout': 30.0, 'packages': [{'name': 'express', 'type': 'npm'}, {'name': 'requests', 'type': 'pypi'}, {'name': 'node', 'type': 'docker'}]}
[10:48:00.538] [REGISTRY-INFO] Registry configurations: {}
[10:48:00.538] [REGISTRY-INFO] Parsing packages config: [{'name': 'express', 'type': 'npm'}, {'name': 'requests', 'type': 'pypi'}, {'name': 'node', 'type': 'docker'}]
[10:48:00.538] [REGISTRY-INFO] Processing package config: {'name': 'express', 'type': 'npm'}
[10:48:00.538] [REGISTRY-INFO] Parsed dict package: express (type: npm, version: None)
[10:48:00.538] [REGISTRY-INFO] Processing package config: {'name': 'requests', 'type': 'pypi'}
[10:48:00.538] [REGISTRY-INFO] Parsed dict package: requests (type: pypi, version: None)
[10:48:00.538] [REGISTRY-INFO] Processing package config: {'name': 'node', 'type': 'docker'}
[10:48:00.538] [REGISTRY-INFO] Parsed dict package: node (type: docker, version: None)
[10:48:00.538] [REGISTRY-INFO] Total packages parsed: 3
[10:48:00.538] [REGISTRY-INFO] Parsed 3 packages for validation
🔍 📋 Configured 6 validators: registry, protocol, capabilities, ping, errors, security
[10:48:00.538] [EXEC-INFO] 🔄 Configured 6 validators: Names: ['registry', 'protocol', 'capabilities', 'ping', 'errors', 'security']
🔍 🚀 Starting validation (6 validators)
[10:48:00.538] [EXEC-INFO] 🔄 Starting validation: Mode: sequential
🔍 🔄 Running registry (1/6)
[10:48:00.538] [VALIDATOR-INFO] 🔍 [registry] STARTING: (1/6)
[10:48:00.538] [REGISTRY-INFO] Starting registry validation with 3 packages
[10:48:00.538] [REGISTRY-INFO] Package source: configuration
[10:48:00.538] [REGISTRY-INFO] Creating HTTP session for registry requests
[10:48:00.539] [REGISTRY-INFO] Processing package 1/3: express (npm)
[10:48:00.539] [REGISTRY-INFO] Checking package express with npm checker
[10:48:00.539] [REGISTRY-INFO] NPM: Checking package 'express' at https://registry.npmjs.org/express
[10:48:00.539] [REGISTRY-INFO] NPM: Making HTTP request to https://registry.npmjs.org/express
[10:48:00.631] [REGISTRY-INFO] NPM: Response status: 200
[10:48:00.649] [REGISTRY-INFO] NPM: Package 'express' exists, found 283 versions
[10:48:00.650] [REGISTRY-INFO] Check result for express: exists=True, error=None
[10:48:00.650] [REGISTRY-INFO] Package express exists
[10:48:00.650] [REGISTRY-INFO] Processing package 2/3: requests (pypi)
[10:48:00.650] [REGISTRY-INFO] Checking package requests with pypi checker
[10:48:00.650] [REGISTRY-INFO] PyPI: Checking package 'requests' at https://pypi.org/pypi/requests/json
[10:48:00.650] [REGISTRY-INFO] PyPI: Making HTTP request to https://pypi.org/pypi/requests/json
[10:48:00.684] [REGISTRY-INFO] PyPI: Response status: 200
[10:48:00.694] [REGISTRY-INFO] PyPI: Package 'requests' exists, found 157 versions
[10:48:00.694] [REGISTRY-INFO] Check result for requests: exists=True, error=None
[10:48:00.694] [REGISTRY-INFO] Package requests exists
[10:48:00.694] [REGISTRY-INFO] Processing package 3/3: node (docker)
[10:48:00.694] [REGISTRY-INFO] Checking package node with docker checker
[10:48:00.694] [REGISTRY-INFO] Docker: Checking official image 'node' at https://hub.docker.com/v2/repositories/library/node
[10:48:00.694] [REGISTRY-INFO] Docker: Making HTTP request to https://hub.docker.com/v2/repositories/library/node
[10:48:00.750] [REGISTRY-INFO] Docker: Response status: 200
[10:48:00.750] [REGISTRY-INFO] Docker: Image 'node' exists
[10:48:00.750] [REGISTRY-INFO] Docker: Getting tags from https://hub.docker.com/v2/repositories/library/node/tags
[10:48:00.864] [REGISTRY-INFO] Docker: Found 10 tags for 'node'
[10:48:00.864] [REGISTRY-INFO] Check result for node: exists=True, error=None
[10:48:00.864] [REGISTRY-INFO] Package node exists
[10:48:00.865] [REGISTRY-INFO] Registry validation completed:
[10:48:00.865] [REGISTRY-INFO]   - Passed: True
[10:48:00.865] [REGISTRY-INFO]   - Errors: 0
[10:48:00.865] [REGISTRY-INFO]   - Warnings: 0
[10:48:00.865] [REGISTRY-INFO]   - Packages found: 3/3
[10:48:00.865] [REGISTRY-INFO]   - Execution time: 0.33s
🔍 ✅ registry: PASSED (0.33s)
[10:48:00.865] [VALIDATOR-INFO] 🔍 [registry] PASSED: Time: 0.33s
🔍 🔄 Running protocol (2/6)
[10:48:00.865] [VALIDATOR-INFO] 🔍 [protocol] STARTING: (2/6)
🔍 📤 Sending MCP request: initialize
🔍 ✅ Initialize request - session already initialized
🔍 📤 Sending notification: notifications/initialized
🔍 ✅ protocol: PASSED (0.00s)
[10:48:00.865] [VALIDATOR-INFO] 🔍 [protocol] PASSED: Time: 0.00s
[10:48:00.865] [VALIDATOR-INFO] 🔍 [protocol] CONTEXT_UPDATED: Server info and capabilities stored for dependent validators
🔍 🔄 Running capabilities (3/6)
[10:48:00.865] [VALIDATOR-INFO] 🔍 [capabilities] STARTING: (3/6)
🔍 📤 Sending MCP request: resources/list
🔍 ❌ MCP request failed: Session terminated
🔍 📤 Sending MCP request: tools/list
🔍 📤 Sending MCP request: prompts/list
🔍 ❌ MCP request failed: Session terminated
🔍 ✅ capabilities: PASSED (0.05s)
[10:48:00.915] [VALIDATOR-INFO] 🔍 [capabilities] PASSED: Time: 0.05s, Warnings: 2
[10:48:00.915] [VALIDATOR-INFO] 🔍 [capabilities] CONTEXT_UPDATED: Discovered items stored: 8 tools, 0 resources, 0 prompts
🔍 🔄 Running ping (4/6)
[10:48:00.915] [VALIDATOR-INFO] 🔍 [ping] STARTING: (4/6)
🔍 📤 Sending MCP request: ping
🔍 ✅ ping: PASSED (0.00s)
[10:48:00.915] [VALIDATOR-INFO] 🔍 [ping] PASSED: Time: 0.00s
🔍 🔄 Running errors (5/6)
[10:48:00.915] [VALIDATOR-INFO] 🔍 [errors] STARTING: (5/6)
🔍 📤 Sending MCP request: invalid_method_that_does_not_exist
🔍 ⚠️ Unsupported method for ClientSession: invalid_method_that_does_not_exist
🔍 ✅ errors: PASSED (0.00s)
[10:48:00.915] [VALIDATOR-INFO] 🔍 [errors] PASSED: Time: 0.00s, Warnings: 1
🔍 🔄 Running security (6/6)
[10:48:00.915] [VALIDATOR-INFO] 🔍 [security] STARTING: (6/6)
🔍 ✅ security: PASSED (0.00s)
[10:48:00.916] [VALIDATOR-INFO] 🔍 [security] PASSED: Time: 0.00s, Warnings: 1
🔍 🏁 Validation completed
[10:48:00.916] [EXEC-INFO] 🔄 Cleaning up transport
🔍 🔄 Closing HTTP transport...
🔍 ✅ MCP client session closed
🔍 ✅ HTTP transport closed successfully
[10:48:00.916] [EXEC-INFO] ✅ Process execution SUCCESS: Transport cleanup completed
🔍 🔄 Closing HTTP transport...
[10:48:00.916] [SUMMARY-INFO]
[10:48:00.916] [SUMMARY-INFO] 📊 Validation Summary:
[10:48:00.916] [SUMMARY-INFO]    Total validators: 6
[10:48:00.916] [SUMMARY-INFO]    Passed: 6
[10:48:00.916] [SUMMARY-INFO]    Failed: 0
[10:48:00.916] [SUMMARY-INFO]    Success rate: 100.0%
[10:48:00.916] [SUMMARY-INFO]    Execution time: 3.24s
✓ Valid: True
⏱ Execution time: 3.24s
📋 Profile: comprehensive
✅ Registry: Passed
✅ Protocol: Passed
    🖥 Server: HTTP MCP Server vunknown
    🔧 Capabilities: tools, resources, prompts, logging
✅ Capabilities: Passed
    🔨 Tools (8): get_mcp_server_version, get_issue, create_issue, get_merge_request, get_merge_request_changes
        ... and 3 more
    ⚠️  resources/list request failed: MCP request failed: Session terminated
    ⚠️  prompts/list request failed: MCP request failed: Session terminated
✅ Ping: Passed
    🏓 Ping: Supported (0.00ms)
✅ Errors: Passed
    ⚠️  Error compliance: invalid method handling validated
    ⚠️  Malformed request error test failed: 'NoneType' object has no attribute 'stdin'
✅ Security: Passed
    🔍 Security: No issues found in 8 tools
    ⚠️  Security analysis failed: list index out of range

⚠️  Warnings:
  - resources/list request failed: MCP request failed: Session terminated
  - prompts/list request failed: MCP request failed: Session terminated
  - Malformed request error test failed: 'NoneType' object has no attribute 'stdin'
  - Security analysis failed: list index out of range

A few thoughts:

  • The server name seems to be hardcoded to "HTTP MCP Server vunknown" - is there some way to populate this like is done with stdio? Otherwise, this information is not really helpful in this case.
  • This server only supports tools capabilities? Why does the Capabilities list (under Protocol) list other kinds of capabilities as well? (resources, prompts, logging)
  • Listing resources and prompts failed. I would consider this not an issue because the server does not advertise any of these in the first place. The capabilities testing should probably only query the capability types advertised by the server.
  • There seems to be some kind of issue on the client side with the malformed request error test, or at the very least, the error message is not very useful (and debug/verbose logging doesn't seem to show anything more).
  • There seems to be an issue with the security analysis (list index out of range) when running against this server - again, not really sure what to do with this message or how to troubleshoot it further.

Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
…xist" was actually correct behavior being logged unnecessarily

Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
…g until the right request ID is received

Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
…quest" when the server silently ignores the malformed request (which is acceptable behavior).

Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
@dmartinol
Copy link
Collaborator Author

Hey @caxu-rh thanks for your comments.
I applied some fixes to handle all the problems you mentioned, now it should look better, please take a look!

Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
@dmartinol
Copy link
Collaborator Author

@caxu-rh do you agree to merge?

Copy link

@caxu-rh caxu-rh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the additional changes. In general I think it's good to go, seems to have been working as expected. Further changes can still be made in additional PRs as needed.

@dmartinol dmartinol merged commit 39c150d into RHEcosystemAppEng:main Nov 4, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants