Skip to content

Conversation

bhosmer-ant
Copy link
Contributor

@bhosmer-ant bhosmer-ant commented Sep 5, 2025

Summary

Implements support for external authorization servers, enabling the MCP server to delegate authentication to a separate OAuth provider. This demonstrates both authorization patterns specified in the MCP specification: integrated (existing) and separate (new).

Key Changes

  • Added configuration for authentication modes
  • Extracted shared authentication logic
  • Implemented server mode switching
  • Created a standalone authorization server
  • Added comprehensive documentation and testing scripts

Architecture

  • Integrated Mode: MCP server acts as its own OAuth authorization server
  • Separate Mode: MCP server delegates to an external authorization server

Testing

  • Both integrated and separate modes can be tested with MCP Inspector
  • Comprehensive end-to-end testing scripts added
  • Verified:
    • Complete OAuth 2.0 + PKCE flows
    • Token introspection
    • Cross-server session management
    • All MCP features working correctly

Backward Compatibility

  • Default mode remains "integrated"
  • Existing APIs and endpoints preserved
  • Shared modules maintain backward compatibility

Implementation Highlights

  • Uses SDK's mcpAuthMetadataRouter
  • Token caching follows OAuth 2.0 best practices
  • Session ownership preserved via Redis

The PR provides a flexible authentication system that supports both integrated and separate authorization server modes.


PR Reviewer's Guide: Dual Authentication Mode Support

Overview

This PR introduces dual authentication mode support to the MCP Example Remote Server, allowing it to operate in two distinct modes:

  1. Integrated Mode (default): MCP server acts as its own OAuth 2.0 authorization server
  2. Separate Mode: MCP server delegates authentication to an external authorization server

This architectural change enables the example server to demonstrate both simple standalone deployments and enterprise integration patterns where organizations use existing OAuth infrastructure (Auth0, Okta, AWS Cognito, etc.).

High-Level Changes

1. Dual Mode Architecture

  • Added configuration system to switch between integrated and separate authentication modes
  • Implemented mode-dependent initialization logic in the main server
  • Created external token verification for separate mode

2. Shared Authentication Logic

  • Extracted common OAuth logic into shared modules used by both servers
  • Implemented Redis namespace isolation to prevent key collisions
  • Centralized token generation, encryption, and storage logic

3. Standalone Authorization Server

  • Created a separate Express application that acts as an OAuth 2.0 server
  • Implements token introspection endpoint (RFC 7662) for external validation
  • Provides complete OAuth flow independent of the MCP server

4. Infrastructure Improvements

  • Consolidated TypeScript build configuration for multi-project setup
  • Added Docker Compose for Redis dependency management
  • Improved build scripts to handle both server and auth-server compilation
  • Fixed npm registry configuration for CI/CD

5. Testing & Documentation

  • Added comprehensive end-to-end test scripts for both modes
  • Created automated OAuth flow verification with PKCE
  • Updated documentation to explain both authentication modes
  • Added architecture diagrams and flow analysis

6. Security Enhancements

  • Added rate limiting to all OAuth endpoints
  • Implemented comprehensive token validation including audience checks
  • Added temporal claim validation (nbf, iat) with clock skew tolerance
  • Improved logging for security audit trails

File-by-File Breakdown

Configuration & Environment Files

.env.integrated, .env.separate (new)

src/config.ts (modified)

  • Purpose: Centralized configuration with mode switching
  • Related to: Change Fix tests to work with ES modules and TypeScript #1 (Dual Mode Architecture)
  • Added AUTH_MODE, AUTH_SERVER_PORT, AUTH_SERVER_URL configuration
  • Added validation to ensure configuration consistency

Shared Authentication Modules

shared/auth-core.ts (new)

shared/redis-auth.ts (new)

shared/types.ts (new)

src/services/auth.ts (modified)

Standalone Authorization Server

auth-server/index.ts (new)

  • Purpose: Complete OAuth 2.0 authorization server
  • Related to: Change Inna example shttp #3 (Standalone Authorization Server)
  • Implements all OAuth endpoints via SDK's mcpAuthRouter
  • Adds /introspect endpoint for token validation (RFC 7662)
  • Includes health check endpoint for monitoring

auth-server/README.md (new)

  • Purpose: Documentation for the standalone auth server
  • Related to: Change Inna example shttp #3 (Standalone Authorization Server)
  • Explains the demonstration purpose and production considerations

auth-server/tsconfig.json (new)

  • Purpose: TypeScript configuration for auth server
  • Related to: Change Inna example shttp #3 (Standalone Authorization Server)
  • Separate build configuration for independent compilation

MCP Server Modifications

src/index.ts (heavily modified)

  • Purpose: Mode-dependent initialization and auth configuration
  • Related to: Change Fix tests to work with ES modules and TypeScript #1 (Dual Mode Architecture)
  • Lines 140-179: Integrated mode setup with local OAuth provider
  • Lines 180-247: Separate mode setup with external auth verification
  • Lines 189-226: Retry logic for connecting to external auth server
  • Lines 228-236: Backwards compatibility OAuth metadata serving
  • Lines 277-280: Conditional fake auth routes (integrated mode only)

src/auth/external-verifier.ts (new)

src/auth/provider.ts (modified)

src/handlers/fakeauth.ts (modified)

  • Purpose: Fake upstream auth for testing
  • Related to: Change draft blog post #5 (Testing & Documentation)
  • Added extensive debug logging for troubleshooting OAuth flows

Infrastructure & Build

tsconfig.json (modified)

package.json (modified)

  • Purpose: Scripts and dependencies
  • Related to: Change Add PRM metadata to www-authenticate #4 (Infrastructure Improvements)
  • Added new scripts: dev:integrated, dev:separate, dev:auth-server, dev:with-separate-auth
  • Added concurrently for running multiple servers
  • Added express-rate-limit for security

docker-compose.yml (new)

package-lock.json (modified)

Testing

scripts/test-integrated-e2e.sh (new)

  • Purpose: End-to-end testing for integrated mode
  • Related to: Change draft blog post #5 (Testing & Documentation)
  • Complete OAuth flow verification with PKCE
  • Tests all MCP features (tools, resources, prompts)
  • Verifies README claims

scripts/test-separate-e2e.sh (new)

  • Purpose: End-to-end testing for separate mode
  • Related to: Change draft blog post #5 (Testing & Documentation)
  • Tests OAuth delegation to external server
  • Verifies token introspection flow
  • Tests cross-server session management

src/services/auth.test.ts (modified)

Documentation

README.md (extensively modified)

  • Purpose: Main documentation
  • Related to: Change draft blog post #5 (Testing & Documentation)
  • Added Quick Start section
  • Added Authentication Modes section explaining both modes
  • Added OAuth Flow Analysis with detailed breakdown
  • Updated architecture diagrams for both modes
  • Added troubleshooting guide

docs/user-id-system.md (modified)

  • Purpose: Authentication flow documentation
  • Related to: Change draft blog post #5 (Testing & Documentation)
  • Added Redis key structure documentation
  • Documented namespace isolation between auth and MCP keys

docs/streamable-http-design.md (modified)

  • Purpose: Transport implementation documentation
  • Related to: Change draft blog post #5 (Testing & Documentation)
  • Updated to reflect dual mode architecture

Key Design Decisions

  1. Redis Namespace Isolation: All auth-related keys use auth: prefix to prevent collisions with MCP session keys, enabling both modes to work consistently.

  2. Backwards Compatibility: In separate mode, the MCP server still serves OAuth metadata for clients that expect it on the resource server, even though it delegates to an external auth server.

  3. Token Introspection: Chose RFC 7662 token introspection over JWT validation to demonstrate stateful token validation patterns common in enterprise environments.

  4. Shared Modules: Extracted common logic to shared/ directory to maintain DRY principle while supporting two distinct deployment modes.

  5. Comprehensive Validation: Added audience validation and temporal claim checks to meet MCP specification requirements for secure token handling.

Testing Recommendations

  1. Integrated Mode: Run npm run test:e2e:integrated to verify the complete flow
  2. Separate Mode: Run npm run test:e2e:separate to test delegation
  3. Manual Testing: Use MCP Inspector with both modes to understand user experience
  4. Unit Tests: Run npm test to verify all components

Migration Impact

  • Non-breaking: Default behavior remains integrated mode
  • Configuration-driven: Mode selection via environment variable
  • Backward compatible: Existing deployments continue to work unchanged

- Updated config.ts with AUTH_MODE, AUTH_SERVER_URL, AUTH_SERVER_PORT
- Added validation for port consistency between URL and PORT variables
- Added comprehensive JSDoc comments for all config variables
- Created .env.integrated and .env.separate config files
- Updated redis.ts to use REDIS_URL from config
- Added .claude/ to .gitignore
- Created shared/types.ts with well-documented type definitions
- Created shared/auth-core.ts with core auth functions (PKCE, tokens, encryption)
- Created shared/redis-auth.ts with Redis operations for auth data
- Updated src/services/auth.ts to use shared modules
- Added TokenIntrospectionResponse interface for Mode 2
- Created ExternalAuthVerifier for token validation with external auth server
- Added mode switching logic to src/index.ts
- Integrated mode uses mcpAuthRouter (current behavior)
- Separate mode uses mcpAuthMetadataRouter and external verifier
- Fake upstream auth routes only available in integrated mode
- Added auth server metadata fetching with error handling
- Corrected imports in external-verifier.ts (OAuthTokenVerifier from provider.js)
- Fixed InvalidTokenError import from errors.js
- Changed logger.warn to logger.info (warn method doesn't exist)
- Removed unused OAuthTokens import from services/auth.ts
- Implemented retry logic (5 attempts, 3 second delay) for auth server connection
- Added token validation caching with 60-second default TTL
- Cache respects token expiration time if provided
- Cache invalidation on 401/403 responses
- Automatic cache cleanup every minute
- Debug logging for cache hits and misses
- Created auth-server/index.ts with full OAuth endpoint support
- Reuses EverythingAuthProvider from main server
- Added custom /oauth/introspect endpoint for token validation
- Includes fake upstream auth routes for user authentication
- Added TypeScript configuration for auth server
- Added comprehensive npm scripts for all modes:
  - dev:integrated, dev:separate, dev:auth-server
  - dev:with-separate-auth (runs both servers)
  - build:auth-server, build:all
- Added concurrently dependency for running multiple servers
- Added detailed README for auth server
- Successfully serves OAuth metadata and health endpoints
- Updated Prerequisites with Redis setup (Docker Compose + OrbStack)
- Added comprehensive Authentication Modes section
- Added Testing with MCP Inspector section with step-by-step guides
- Updated Features to highlight dual mode support
- Updated Configuration with new environment variables
- Updated Development Commands with all new npm scripts
- Added architecture diagrams for both modes
- Referenced auth-server/README.md for detailed auth server info

All phases now completed - Mode 2 implementation ready for use!
Copy link
Member

@pcarleton pcarleton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, few small tweaks

* fix: add /mcp to endpoints in misc README.mds
* improve architecture interaction diagrams in README.md
* add "OAuth Flow Analysis" section in README.md
- Remove /oauth prefix from introspect endpoint to match SDK pattern
- Fix auth-server endpoints: /introspect not /oauth/introspect
- Update external-verifier to call correct introspect endpoint
- Add 30-day expiry to client registration (REDIS_EXPIRY_TIMES.CLIENT_REGISTRATION)
- Remove inaccurate scratch/ directory references from README
- Add backlinks from auth-server/README.md to main README sections
- Verified all implementation details against running servers and Redis data
- Fix server hanging issue: incomplete HTTP responses in shttp.ts
- Add proper JSON-RPC error responses per MCP specification
- Fix PKCE challenge encoding: use base64url not standard base64
- Add comprehensive e2e scripts with OAuth flow and feature testing:
  - scripts/test-integrated-e2e.sh: Tests integrated mode
  - scripts/test-separate-e2e.sh: Tests separate mode
- Scripts verify all README claims: 7 tools, 100 resources (paginated), 3 prompts
- Add environment variable support and prerequisite checking
- Add TODO documentation for Streamable HTTP implementation choices
- Update README with table of contents and scripts documentation
- Update project structure to include scripts directory

Scripts successfully verify:
✅ Complete OAuth 2.0 + PKCE flows in both modes
✅ Token introspection and cross-server session management
✅ All MCP features working correctly
✅ README accuracy (tool/resource counts verified programmatically)
- Update streamable-http-design.md with correct file references
- Reflect current dual transport architecture (SSE + Streamable HTTP)
- Update file paths to match actual implementation structure
- Clarify stateful session management approach
- Remove outdated research content, focus on current implementation
- Add express-rate-limit to custom endpoints for security:
  - /introspect: 100 requests per 5 minutes
  - /fakeupstreamauth/*: 20 requests per minute
  - Static files: 25 requests per 10 minutes
- Add automated e2e testing npm scripts using concurrently:
  - npm run test:e2e:integrated: Auto-starts server and runs test
  - npm run test:e2e:separate: Auto-starts both servers and runs test
- Update README with automated testing approach documentation
- Update docs/streamable-http-design.md with current file structure
- Remove redundancy in README e2e testing documentation
- Verified: All tests, linting, and e2e scripts pass with rate limiting
- Apply staticFileRateLimit to /mcp-logo.png route in auth server
- Addresses remaining check failure about missing rate limiting
- Matches rate limiting pattern used in main server
- Add missing staticFileRateLimit definition in auth-server/index.ts
- Resolves compilation error in auth server build
- Addresses PR check failure about missing rate limiting on line 123
@bhosmer-ant bhosmer-ant marked this pull request as ready for review September 12, 2025 04:46
bhosmer-ant and others added 3 commits September 12, 2025 00:48
- Make 'npm run build' build both main server and auth server
- Remove redundant build:auth-server and build:all scripts
- Update README to reflect simplified build command
- Ensures compilation errors in auth server are caught by default build
The package-lock.json contained references to Anthropic's internal
artifactory registry which GitHub Actions cannot access, causing
npm ci to fail. Regenerated with public registry.npmjs.org.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

// Serve resource metadata only (not auth endpoints)
app.use(mcpAuthMetadataRouter({
oauthMetadata: authMetadata,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thnk we should only be serving the .well-known/oauth-authorization-server endpoint on integrated mode, but I think the code here is about populating for separate mode

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be addressed now, but also worth validating with @pcarleton

Remove incorrect mcpAuthMetadataRouter call when running in separate mode.
The resource server should not serve OAuth metadata endpoints - only the
auth server should provide these endpoints.
Use single tsconfig.json for both main server and auth-server to avoid
nested directory issues during build. Update e2e test scripts to run
directly instead of using concurrently.
Add logging to track authorization flow steps and token introspection
to help diagnose issues with separate mode authentication.
Add state parameter round-trip validation to prevent CSRF attacks.
Document each OAuth step including PKCE generation and token exchange.
Kill existing servers before starting new ones to avoid conflicts.
bhosmer-ant and others added 4 commits September 15, 2025 02:33
Fixes MCP Inspector connection by providing OAuth discovery endpoint
that points to the external auth server at http://localhost:3001
Keep serving .well-known/oauth-authorization-server from MCP server
in separate mode as some clients may expect to find it there
Enhance token validation in separate mode to fully comply with MCP specification:
- Validate audience (aud) claim to ensure tokens are issued for this specific MCP server
- Validate temporal claims (nbf, iat) with appropriate clock skew tolerance
- Add configurable canonical URI for audience validation
- Improve logging for validation failures

These changes prevent token passthrough attacks and ensure tokens are
properly scoped to the intended resource server, as required by the
MCP OAuth 2.0 specification.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
Copy link
Member

@pcarleton pcarleton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

from irl: we'll deploy integrated mode, we could do separate in the future.

one thing to note is that b/c we do the oauth metadata endpoint for backwards compatibility, the two modes are somewhat difficult to distinguish.

@dylang
Copy link

dylang commented Sep 24, 2025

Hi! I know is already approved, so I'm happy to move this feedback to a new ticket!

I've been using this branch for the last few days with external auth. It took way long than I was hoping to get it working, here's some reasons:

  1. The issue I was having with external auth was readMcpInstallation never had the data despite the auth flow seemingly going smoothly, so it always returned 401.
  2. I was confused by EverythingOAuthClientsStore in provider.ts - Are these and services/auth.ts only for internal? Could we rename them to make that clear?
  3. Code named "fake*" like fakeauth - I normally associate "fake*" with mocks for tests. Are they for testing, something we should replace, or useable for internal-auth?
  4. It would be helpful to list some MCP clients that this has been verified to work in. For example, I have external auth working with Gemini CLI and Github Copiliot, but it fails with MCP Inspector. I don't know why et, it could be my own code.

Huge thank you for the work you've done on this project! You've saved me countless hours and helped me understand oauth again!

package.json Outdated
"lint": "eslint src/",
"test": "NODE_OPTIONS=--experimental-vm-modules jest"
"copy-static": "mkdir -p dist/src/static && cp -r src/static/* dist/src/static/",
"lint": "eslint src/ auth-server/",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's also a shared directory now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah good catch, thanks!

@bhosmer-ant
Copy link
Contributor Author

@dylang super appreciate you trying this out! Will go through all your feedback shortly - I'd rather fix now than kick the can down the road, but if I run into something that's hard to address I'll call it out so you can raise it separately.

The auth server's /introspect endpoint now correctly sets the 'aud' field
to the resource server URL (BASE_URI) instead of the client ID. This ensures
proper audience validation when the MCP server verifies tokens in separate mode.

- Import BASE_URI in auth server
- Set aud to BASE_URI in introspection response
- Fixes "Token was not issued for this resource server" error

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@bhosmer-ant bhosmer-ant merged commit 22b6268 into main Sep 26, 2025
4 checks passed
@bhosmer-ant
Copy link
Contributor Author

bhosmer-ant commented Sep 26, 2025

@dylang I've gone ahead and pushed this with a couple of minor fixes/tweaks rather than churning it, so most of your callouts are still pending - some responses and followup requests:

The issue I was having with external auth was readMcpInstallation never had the data despite the auth flow seemingly going smoothly, so it always returned 401.

Any chance you have a repro? It works for me using Inspector and the new e2e test, but more datapoints definitely welcome.

I was confused by EverythingOAuthClientsStore in provider.ts - Are these and services/auth.ts only for internal? Could we rename them to make that clear?

Yeah, only for internal - separate-mode stuff should only be in auth-server/ and shared/. [edit: "should be" in shared but I was wrong, these are used in both the integrated auth and the separate server. Will reorg and possibly rename for clarification]

Code named "fake*" like fakeauth - I normally associate "fake*" with mocks for tests. Are they for testing, something we should replace, or useable for internal-auth?

Agree that this could be confusing - something like "demo" would probably be better. A production server acting as its own auth server would probably need to swap this out for something more industrial-grade.

It would be helpful to list some MCP clients that this has been verified to work in. For example, I have external auth working with Gemini CLI and Github Copiliot, but it fails with MCP Inspector. I don't know why et, it could be my own code.

Yes, this would be helpful. If you want to start a new section in the README and add the verifications you've done with a little detail, that would be a great start!

And re failing with Inspector, if you can put together a repro in a new issue, that would be super helpful.

Thanks again for kicking the tires so thoroughly on this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants