docs: streamline auth documentation and add scopes.md references

- Simplify docs/auth.md Cognito section to reference detailed guides
- Add scopes.md references throughout README.md for FGAC topics
- Improve documentation navigation for authentication and access control

Author: aarora79

README.md

@@ -56,7 +56,7 @@ You can deploy the gateway and registry on Amazon EC2 or Amazon EKS for producti
 ## What's New
 
 * **IdP Integration with Amazon Cognito:** Complete identity provider integration supporting both user identity and agent identity modes. See [detailed Cognito setup guide](docs/cognito.md) for configuration instructions.
-* **Fine-Grained Access Control (FGAC) for MCP servers and tools:** Granular permissions system allowing precise control over which agents can access specific servers and tools
+* **Fine-Grained Access Control (FGAC) for MCP servers and tools:** Granular permissions system allowing precise control over which agents can access specific servers and tools. See [detailed FGAC documentation](docs/scopes.md) for scope configuration and access control setup.
 * **Integration with [Strands Agents](https://github.com/strands-agents/sdk-python):** Enhanced agent capabilities with the Strands SDK
 * **Dynamic tool discovery and invocation:** AI agents can autonomously discover and execute specialized tools beyond their initial capabilities using semantic search with FAISS indexing and sentence transformers. This breakthrough feature enables agents to handle tasks they weren't originally designed for by intelligently matching natural language queries to the most relevant MCP tools across all registered servers. [Learn more about Dynamic Tool Discovery →](docs/dynamic-tool-discovery.md)
 * **[Installation on EKS](#installation-on-eks):** Deploy on Kubernetes for production environments
@@ -222,7 +222,7 @@ flowchart TB
 
 *   **External API Keys (Optional):** One of the example MCP servers uses the [`Polygon`](https://polygon.io/stocks) API for stock ticker data. Get an API key from [here](https://polygon.io/dashboard/signup?redirect=%2Fdashboard%2Fkeys). The server will still start without the API key but you will get a 401 Unauthorized error when using the tools provided by this server.
 
-*   **Authentication Setup:** Setup authentication using Amazon Cognito as per instructions [here](docs/auth.md). For detailed Cognito configuration, see the [Cognito setup guide](docs/cognito.md).
+*   **Authentication Setup:** Setup authentication using Amazon Cognito as per instructions [here](docs/auth.md). For detailed Cognito configuration, see the [Cognito setup guide](docs/cognito.md). For Fine-Grained Access Control (FGAC) configuration, see the [scopes documentation](docs/scopes.md).
 
 ## Installation
 
@@ -288,6 +288,7 @@ The deployment includes these containers:
    - Access permissions will be based on the Cognito group you are a member of
    - Provides fine-grained access control based on your organizational roles
    - For detailed Cognito setup instructions, see [docs/cognito.md](docs/cognito.md)
+   - For Fine-Grained Access Control (FGAC) configuration and scope management, see [docs/scopes.md](docs/scopes.md)
 
    **Option 2 - Username/Password (Testing Only):**
    - Use the traditional login with:

docs/auth.md

@@ -214,128 +214,8 @@ The above implementation provides an OAuth compliant way to MCP security without
 
 ## Amazon Cognito based reference implementation
 
-This section discusses the reference implementation using Amazon Cognito as the IdP, supporting both Machine-to-Machine (M2M) and Session Cookie authentication methods.
-
->For comprehensive Cognito setup instructions, see [`docs/cognito.md`](cognito.md) which covers both user identity and agent identity authentication modes with step-by-step configuration guides.
-
-### Key Components
-
-#### 1. Auth Server (`auth_server/server.py`)
-The auth server provides dual authentication support:
-- **Primary Check**: Session cookie validation using `itsdangerous.URLSafeTimedSerializer`
-- **Fallback**: JWT token validation with Cognito
-- **Group Mapping**: Maps Cognito groups to MCP scopes via `scopes.yml` configuration
-  - Both M2M and session cookie auth use the same scope definitions
-
-#### 2. CLI Authentication Tool (`agents/cli_user_auth.py`)
-A standalone tool for user-based authentication:
-- Implements OAuth 2.0 PKCE flow with Cognito hosted UI
-- Opens browser for user authentication
-- Runs local callback server on port 8080
-- Creates session cookie compatible with registry format
-- Saves to `~/.mcp/session_cookie` with secure permissions (0600)
-
-#### 3. Agent (`agents/agent.py`)
-The agent supports both authentication methods:
-- `--use-session-cookie` flag for session-based auth
-- `--session-cookie-file` parameter (default: `~/.mcp/session_cookie`)
-- Maintains full backward compatibility with M2M authentication
-- Automatically includes appropriate headers based on auth method
-
-### 1. Machine-to-Machine Authentication
-
-Cognito supports machine-to-machine authentication, enabling Agents to have their own identity separate from user identity.
-
-#### Implementation Details:
-- Reference: [AWS Blog on Machine-to-Machine Authentication](https://aws.amazon.com/blogs/mt/configuring-machine-to-machine-authentication-with-amazon-cognito-and-amazon-api-gateway-part-2/)
-- Agents are treated as App Clients (Cognito terminology)
-- MCP Server(s) function as resource servers
-
-#### Authentication Flow:
-Run the Agent with the following command:
-```{.bash}
-python agents/agent.py
-```
-1. Copy `agents/.env.template` to `agents/.env.agent` and set the environment variables (`COGNITO_CLIENT_ID`, `COGNITO_CLIENT_SECRET`, `COGNITO_USER_POOL_ID`) as appropriate for your setup. For detailed Cognito configuration steps, see [`docs/cognito.md`](cognito.md).
-1. Agent startup:
-   - Configured with client ID, client secret, and a set of scopes. _Each agent is an App Client in a Cognito user pool_.
-   - Requests scopes (e.g., MCP Registry with tool finder and basic MCP servers)
-1. Cognito issues a JWT token
-1. Agent includes the JWT token in MCP headers
-1. Auth server on Nginx side:
-   - Retrieves JWT token
-   - Calls Cognito to validate token and get allowed scopes
-   - Returns 200 or 403 based on:
-     - URL (MCP server)
-     - Payload (Tools)
-     - Agent's allowed scopes
-
-#### Advantages
-1. Leverages existing Cognito user identities and groups
-1. No need to manage separate M2M credentials for user-initiated actions
-1. Maintains user context throughout the session
-1. Compatible with existing web-based authentication flow
-1. Auth server handles both authentication methods transparently
-
-### 2. Session Cookie Authentication
-
-Session cookie authentication enables agents to act on behalf of users, using their Cognito identity and group memberships for authorization.
-
-#### Implementation Components
-
-##### a. CLI Authentication Tool (`agents/cli_user_auth.py`)
-
-The CLI tool handles the OAuth flow with Cognito and saves the session cookie locally:
-
-```bash
-# Run the CLI authentication tool
-python agents/cli_user_auth.py
-
-# This will:
-# 1. Open your browser to Cognito hosted UI
-# 2. After login, capture the authorization code
-# 3. Exchange code for user information
-# 4. Create and save session cookie to ~/.mcp/session_cookie
-```
-
-Required environment variables:
-- `COGNITO_USER_POOL_ID`: Your Cognito user pool id 
-- `COGNITO_CLIENT_ID`: OAuth client ID configured for PKCE flow
-- `SECRET_KEY`: Must match the registry's SECRET_KEY for cookie compatibility
-
-##### b. Agent with Session Cookie Support
-
-Copy `agents/.env.template` to `agents/.env.user` and set the environment variables (`COGNITO_CLIENT_ID`, `COGNITO_CLIENT_SECRET`, `COGNITO_USER_POOL_ID`, `SECRET_KEY`) as appropriate for your setup. For detailed Cognito configuration steps, see [`docs/cognito.md`](cognito.md).
-
-The agent (`agents/agent.py`) supports session cookie authentication:
-
-```bash
-# Use agent with session cookie
-python agent.py \
-  --use-session-cookie \
-  --message "What time is it in Tokyo?" \
-  --mcp-registry-url http://localhost/mcpgw/sse
-```
+For comprehensive setup instructions and detailed configuration of Amazon Cognito as the Identity Provider, see the detailed documentation in [`docs/cognito.md`](cognito.md) which covers both user identity and agent identity authentication modes with step-by-step configuration guides.
 
-Key features:
-- `--use-session-cookie`: Enable session cookie authentication mode
-- `--session-cookie-file`: Cookie file path (default: `~/.mcp/session_cookie`)
-- Automatically reads cookie and includes in request headers
-- Falls back to M2M if session cookie flag not provided
-
-##### c. Auth Server
-
-The auth server validates session cookies alongside JWT tokens:
-- Checks for `mcp_gateway_session` cookie in request headers
-- Validates cookie signature using `itsdangerous.URLSafeTimedSerializer`
-- Maps Cognito groups to MCP scopes using `scopes.yml` configuration:
-  - Configuration-driven mapping ensures consistency with M2M authentication
-  - Single source of truth for all permission definitions
-- Falls back to JWT validation if no valid cookie found
-
-#### Advantages:
-- Simpler implementation compared to user-based authentication
-- Enables fine-grained control over Agent permissions
-- Facilitates secure machine-to-machine communication within the MCP ecosystem
+For information about Fine-Grained Access Control (FGAC) including scope configuration, group mappings, and permission management, see [`docs/scopes.md`](scopes.md).
 
 By implementing these enhancements, we can significantly improve the security, scalability, and flexibility of our MCP authentication and authorization system.