tsdown + cli with commander (#17)

* feat: tsdown

* fix: formatting

* automate version updates in cli

* fix: docs

* scenarios

* fix: commander deps

Author: mattzcarey

.gitignore

@@ -1,3 +1,4 @@
 node_modules
 results/
-lefthook-local.yml
+lefthook-local.yml
+dist

README.md

@@ -1,6 +1,6 @@
 # MCP Conformance Test Framework
 
-A framework for testing MCP (Model Context Protocol) client implementations against the specification.
+A framework for testing MCP (Model Context Protocol) client and server implementations against the specification.
 
 > [!WARNING] This repository is a work in progress and is unstable. Join the conversation in the #conformance-testing-wg in the MCP Contributors discord.
 
@@ -9,56 +9,110 @@ A framework for testing MCP (Model Context Protocol) client implementations agai
 ### Testing Clients
 
 ```bash
-npm install
-npm run start -- --command "tsx examples/clients/typescript/test1.ts" --scenario initialize
+npx @modelcontextprotocol/conformance client --command "tsx examples/clients/typescript/test1.ts" --scenario initialize
 ```
 
 ### Testing Servers
 
 ```bash
-npm run test:server -- --server-url http://localhost:3000/mcp --all
+# Run all server scenarios (default)
+npx @modelcontextprotocol/conformance server --url http://localhost:3000/mcp
+
+# Run a single scenario
+npx @modelcontextprotocol/conformance server --url http://localhost:3000/mcp --scenario server-initialize
+```
+
+### List Available Scenarios
+
+```bash
+npx @modelcontextprotocol/conformance list
 ```
 
 ## Overview
 
-The conformance test framework validates MCP client implementations by:
+The conformance test framework validates MCP implementations by:
+
+**For Clients:**
 
 1. Starting a test server for the specified scenario
 2. Running the client implementation with the test server URL
 3. Capturing MCP protocol interactions
 4. Running conformance checks against the specification
 5. Generating detailed test results
 
+**For Servers:**
+
+1. Connecting to the running server as an MCP client
+2. Sending test requests and capturing responses
+3. Running conformance checks against server behavior
+4. Generating detailed test results
+
 ## Usage
 
+### Client Testing
+
 ```bash
-npm run start -- --command "<client-command>" --scenario <scenario-name>
+npx @modelcontextprotocol/conformance client --command "<client-command>" --scenario <scenario-name> [options]
 ```
 
+**Options:**
+
 - `--command` - The command to run your MCP client (can include flags)
 - `--scenario` - The test scenario to run (e.g., "initialize")
+- `--timeout` - Timeout in milliseconds (default: 30000)
+- `--verbose` - Show verbose output
 
 The framework appends the server URL as the final argument to your command.
 
+### Server Testing
+
+```bash
+npx @modelcontextprotocol/conformance server --url <url> [--scenario <scenario>]
+```
+
+**Options:**
+
+- `--url` - URL of the server to test
+- `--scenario <scenario>` - Test scenario to run (e.g., "server-initialize". Runs all available scenarios by default
+
 ## Test Results
 
-Results are saved to `results/<scenario>-<timestamp>/`:
+**Client Testing** - Results are saved to `results/<scenario>-<timestamp>/`:
 
 - `checks.json` - Array of conformance check results with pass/fail status
 - `stdout.txt` - Client stdout output
 - `stderr.txt` - Client stderr output
 
+**Server Testing** - Results are saved to `results/server-<scenario>-<timestamp>/`:
+
+- `checks.json` - Array of conformance check results with pass/fail status
+
 ## Example Clients
 
 - `examples/clients/typescript/test1.ts` - Valid MCP client (passes all checks)
 - `examples/clients/typescript/test-broken.ts` - Invalid client missing required fields (fails checks)
 
 ## Available Scenarios
 
+### Client Scenarios
+
 - **initialize** - Tests MCP client initialization handshake
   - Validates protocol version
   - Validates clientInfo (name and version)
   - Validates server response handling
+- **tools-call** - Tests tool invocation
+- **auth/basic-dcr** - Tests OAuth Dynamic Client Registration flow
+- **auth/basic-metadata-var1** - Tests OAuth with authorization metadata
+
+### Server Scenarios
+
+Run `npx @modelcontextprotocol/conformance list --server` to see all available server scenarios, including:
+
+- **server-initialize** - Tests server initialization and capabilities
+- **tools-list** - Tests tool listing endpoint
+- **tools-call-\*** - Various tool invocation scenarios
+- **resources-\*** - Resource management scenarios
+- **prompts-\*** - Prompt management scenarios
 
 ## Architecture
 
@@ -67,8 +121,13 @@ See `src/runner/DESIGN.md` for detailed architecture documentation.
 ### Key Components
 
 - **Runner** (`src/runner/`) - Orchestrates test execution and result generation
-- **Scenarios** (`src/scenarios/`) - Test scenarios with custom server implementations
-- **Checks** (`src/checks.ts`) - Conformance validation functions
+  - `client.ts` - Client testing implementation
+  - `server.ts` - Server testing implementation
+  - `utils.ts` - Shared utilities
+  - `index.ts` - Public API exports
+- **CLI** (`src/index.ts`) - Command-line interface using Commander.js
+- **Scenarios** (`src/scenarios/`) - Test scenarios with expected behaviors
+- **Checks** (`src/checks/`) - Conformance validation functions
 - **Types** (`src/types.ts`) - Shared type definitions
 
 ## Adding New Scenarios

SERVER_REQUIREMENTS.md

@@ -1022,8 +1022,8 @@ MCP Conformance Test Server running on http://localhost:3000
 
 ```bash
 # Single test
-npm run test:server -- --server-url http://localhost:3000/mcp --scenario server-initialize
+npm run start -- server --url http://localhost:3000/mcp --scenario server-initialize
 
 # All tests
-npm run test:server -- --server-url http://localhost:3000/mcp --all
+npm run start -- server --url http://localhost:3000/mcp
 ```

examples/servers/typescript/README.md

@@ -147,8 +147,7 @@ This server implements the MCP Server Conformance Requirements specified in `../
 To run conformance tests against this server:
 
 ```bash
-cd ../../../
-npm run test:server -- --server-url http://localhost:3000/mcp --all
+npx @modelcontextprotocol/conformance server --url http://localhost:3000/mcp
 ```
 
 ## Implementation Notes