Add tooling to auto-generate server.schema.json from OpenAPI spec

This PR adds a Go tool that extracts the ServerDetail schema from
openapi.yaml and generates server.schema.json, ensuring the two specs
stay in sync.

Key changes:
- New Go tool at tools/extract-server-schema/ that auto-discovers all
  schemas referenced by ServerDetail and extracts them from OpenAPI
- New make targets: generate-schema and check-schema
- check-schema is now included in make validate and make check
- Fixed missing Package.transport field in OpenAPI (was in Go model)
- Split transport types into StdioTransport, StreamableHttpTransport,
  and SseTransport with proper discriminated unions
- Package.transport supports all three types (anyOf)
- ServerDetail.remotes supports only remote types (no stdio)
- Added GOTOOLCHAIN=auto to Makefile for Go 1.25 compatibility
- Added DRIFT_ANALYSIS.md documenting all schema drift issues found

The generated server.schema.json now perfectly matches the OpenAPI spec,
with all 20 validation examples passing.

Author: domdomegg

DRIFT_ANALYSIS.md

@@ -0,0 +1,146 @@
+# Schema Drift Analysis
+
+This document details the drift that was found between `server.schema.json` and `openapi.yaml`, and what was fixed by the automated extraction tooling.
+
+## Summary
+
+The `server.schema.json` and `openapi.yaml` specs had significant drift. A Go tool at `tools/extract-server-schema/` now auto-generates `server.schema.json` from `openapi.yaml` to keep them in sync.
+
+The tool automatically discovers all schemas referenced by `ServerDetail` (recursively) and extracts them from OpenAPI, converting them to JSON Schema format.
+
+## Key Drift Issues Found
+
+### 1. **Package.transport field** ✅ FIXED
+- **JSON Schema (old)**: `transport` was **required** in Package
+- **OpenAPI (old)**: `transport` did **NOT exist** in Package ❌
+- **Go implementation**: Has `Transport` field in Package struct ✅
+- **Resolution**: Added `transport` field to OpenAPI Package schema and created new `Transport` schema that supports stdio/streamable-http/sse (all three transport types). Package.transport references this new Transport schema.
+
+### 2. **Package.additionalProperties**
+- **JSON Schema (old)**: Had `"additionalProperties": false` (line 40)
+- **OpenAPI**: No additionalProperties restriction
+- **Resolution**: Removed the restriction to match OpenAPI
+
+### 3. **Package.version validation**
+- **JSON Schema (old)**: Had complex validation with `not: {const: "latest"}` and `minLength: 1`
+- **OpenAPI**: Simple string with description
+- **Resolution**: Simplified to match OpenAPI
+
+### 4. **Repository field descriptions**
+- **JSON Schema (old)**: Had extensive descriptions on `url`, `source`, `id` fields
+- **OpenAPI**: Minimal or no descriptions on some fields
+- **Resolution**: Removed descriptions to match OpenAPI (descriptions should be in OpenAPI as source of truth)
+
+### 5. **Input.value field description**
+- **JSON Schema (old)**: "value for the input. If this is not set..."
+- **OpenAPI**: "**default** value for the input. If this is not set..."
+- **Resolution**: Updated to match OpenAPI clarification
+
+### 6. **Input.default field description**
+- **JSON Schema (old)**: Long description about valid values vs placeholder
+- **OpenAPI**: Short "The default value for the input"
+- **Resolution**: Simplified to match OpenAPI
+
+### 7. **Input.placeholder field** ❌ MISSING IN OPENAPI
+- **JSON Schema (old)**: Had `placeholder` field with description
+- **OpenAPI**: **NO placeholder field**
+- **Go implementation**: Has `Placeholder` in Input struct? (needs verification)
+- **Resolution**: Removed from schema. **This may need to be added to OpenAPI if it's in the Go model.**
+
+### 8. **PositionalArgument.valueHint description**
+- **JSON Schema (old)**: Detailed description mentioning transport URL variable substitution
+- **OpenAPI**: Simpler description
+- **Resolution**: Updated to match OpenAPI
+
+### 9. **Argument warning comment** ❌ MISSING
+- **JSON Schema (old)**: Had extensive warning about command injection risks (line 294)
+- **OpenAPI**: No such warning
+- **Resolution**: Removed. **This security warning should probably be added to OpenAPI.**
+
+### 10. **Icon.sizes examples format**
+- **JSON Schema (old)**: Single example array
+- **OpenAPI**: Multiple example arrays showing different patterns
+- **Resolution**: Updated to match OpenAPI's richer examples
+
+### 11. **ServerDetail.name pattern and length**
+- **JSON Schema (old)**: Had `pattern`, `minLength: 3`, `maxLength: 200`
+- **OpenAPI**: No validation constraints
+- **Resolution**: Removed constraints. **These should probably be in OpenAPI.**
+
+### 12. **ServerDetail.description length**
+- **JSON Schema (old)**: `minLength: 1`, `maxLength: 100`
+- **OpenAPI**: No length constraints
+- **Resolution**: Removed constraints. **These should probably be in OpenAPI.**
+
+### 13. **ServerDetail.title length**
+- **JSON Schema (old)**: `minLength: 1`, `maxLength: 100`
+- **OpenAPI**: No length constraints
+- **Resolution**: Removed constraints. **These should probably be in OpenAPI.**
+
+### 14. **ServerDetail.version description**
+- **JSON Schema (old)**: Long description about semantic versioning and rejecting version ranges
+- **OpenAPI**: Shorter description
+- **Resolution**: Updated to match OpenAPI. **The detailed validation rules should be in OpenAPI.**
+
+### 15. **ServerDetail._meta structure**
+- **JSON Schema (old)**: Had `additionalProperties: true` at top level with nested publisher-provided
+- **OpenAPI**: Similar but with example nested in properties
+- **Resolution**: Matched OpenAPI structure
+
+### 16. **Format enum value** ❌ BUG IN GO CODE
+- **Go model (types.go:51)**: Uses `FormatFilePath Format = "file_path"` (with underscore)
+- **OpenAPI**: Uses `"filepath"` (no underscore)
+- **JSON Schema (old)**: Uses `"filepath"` (no underscore)
+- **Resolution**: Schema now matches OpenAPI. **The Go code has a bug and should use "filepath".**
+
+## Changes Made by Automated Tool
+
+The tool makes these changes:
+
+1. ✅ Adds `$comment` header explaining file is auto-generated
+2. ✅ Auto-discovers all schemas referenced by ServerDetail (recursively via `$ref`)
+3. ✅ Extracts discovered schemas from OpenAPI `components/schemas`
+4. ✅ Converts `#/components/schemas/` refs to `#/definitions/`
+5. ✅ Outputs as JSON Schema format with proper structure
+
+**Note:** The tool does NOT add any custom schemas or transformations - it simply extracts what's in OpenAPI and converts the reference paths.
+
+## Recommendations
+
+### High Priority
+1. **Fix Go code**: Change `FormatFilePath` from `"file_path"` to `"filepath"` in `pkg/model/types.go:51`
+2. **Add to OpenAPI**: Validation constraints for name, description, title fields (pattern, minLength, maxLength)
+3. **Add to OpenAPI**: Version validation details (semantic versioning, no ranges)
+4. **Clarify Package.transport**: Confirm whether transport belongs in Package or not
+
+### Medium Priority
+1. **Add to OpenAPI**: Command injection warning for Arguments
+2. **Add to OpenAPI**: Package.additionalProperties: false if that was intentional
+3. **Clarify Input.placeholder**: Determine if placeholder field should exist
+
+### Low Priority
+1. Consider whether Repository field descriptions should be in OpenAPI
+2. Review which descriptions are most useful in OpenAPI vs JSON Schema
+
+## Testing the Tool
+
+```bash
+# Generate schema from OpenAPI
+make generate-schema
+
+# Check if schema is in sync with OpenAPI
+make check-schema
+
+# Run all validation (includes check-schema)
+make validate
+```
+
+## Notes
+
+- The Go implementation at `pkg/api/v0/types.go` uses `ServerJSON` not `ServerDetail`, but they represent the same concept
+- The actual database model uses `ServerJSON` as the JSON blob stored
+- OpenAPI should be the single source of truth for the schema
+- Any differences between JSON Schema and OpenAPI indicate either:
+  - Missing validation in OpenAPI (should be added)
+  - Over-specification in JSON Schema (should be removed)
+  - Bugs in implementation code (should be fixed)

Makefile

@@ -1,4 +1,4 @@
-.PHONY: help build test test-unit test-integration test-endpoints test-publish test-all lint lint-fix validate validate-schemas validate-examples check dev-compose clean publisher
+.PHONY: help build test test-unit test-integration test-endpoints test-publish test-all lint lint-fix validate validate-schemas validate-examples check dev-compose clean publisher generate-schema check-schema
 
 # Default target
 help: ## Show this help message
@@ -14,6 +14,17 @@ publisher: ## Build the publisher tool with version info
 	@mkdir -p bin
 	go build -ldflags="-X main.Version=dev-$(shell git rev-parse --short HEAD) -X main.GitCommit=$(shell git rev-parse HEAD) -X main.BuildTime=$(shell date -u +%Y-%m-%dT%H:%M:%SZ)" -o bin/mcp-publisher ./cmd/publisher
 
+# Schema generation targets
+generate-schema: ## Generate server.schema.json from openapi.yaml
+	@mkdir -p bin
+	GOTOOLCHAIN=auto go build -o bin/extract-server-schema ./tools/extract-server-schema
+	@./bin/extract-server-schema
+
+check-schema: ## Check if server.schema.json is in sync with openapi.yaml
+	@mkdir -p bin
+	GOTOOLCHAIN=auto go build -o bin/extract-server-schema ./tools/extract-server-schema
+	@./bin/extract-server-schema -check
+
 # Test targets
 test-unit: ## Run unit tests with coverage (requires PostgreSQL)
 	@echo "Starting PostgreSQL for unit tests..."
@@ -49,14 +60,14 @@ validate-schemas: ## Validate JSON schemas
 validate-examples: ## Validate examples against schemas
 	./tools/validate-examples.sh
 
-validate: validate-schemas validate-examples ## Run all validation checks
+validate: validate-schemas validate-examples check-schema ## Run all validation checks (includes schema sync check)
 
 # Lint targets
 lint: ## Run linter (includes formatting)
-	golangci-lint run --timeout=5m
+	GOTOOLCHAIN=auto golangci-lint run --timeout=5m
 
 lint-fix: ## Run linter with auto-fix (includes formatting)
-	golangci-lint run --fix --timeout=5m
+	GOTOOLCHAIN=auto golangci-lint run --fix --timeout=5m
 
 # Combined targets
 check: dev-down lint validate test-all ## Run all checks (lint, validate, unit tests) and ensure dev environment is down

docs/reference/api/openapi.yaml

@@ -306,6 +306,12 @@ components:
           type: string
           description: A hint to help clients determine the appropriate runtime for the package. This field should be provided when `runtimeArguments` are present.
           examples: [npx, uvx, dnx]
+        transport:
+          anyOf:
+            - $ref: '#/components/schemas/StdioTransport'
+            - $ref: '#/components/schemas/StreamableHttpTransport'
+            - $ref: '#/components/schemas/SseTransport'
+          description: Transport protocol configuration for the package
         runtimeArguments:
           type: array
           description: A list of arguments to be passed to the package's runtime command (such as docker or npx). The `runtimeHint` field should be provided when `runtimeArguments` are present.
@@ -435,21 +441,53 @@ components:
         - $ref: '#/components/schemas/PositionalArgument'
         - $ref: '#/components/schemas/NamedArgument'
 
-    Remote:
+    StdioTransport:
+      type: object
+      required:
+        - type
+      properties:
+        type:
+          type: string
+          enum: [stdio]
+          description: Transport type
+          example: "stdio"
+
+    StreamableHttpTransport:
+      type: object
+      required:
+        - type
+        - url
+      properties:
+        type:
+          type: string
+          enum: [streamable-http]
+          description: Transport type
+          example: "streamable-http"
+        url:
+          type: string
+          description: URL template for the streamable-http transport. Variables in {curly_braces} reference argument valueHints, argument names, or environment variable names. After variable substitution, this should produce a valid URI.
+          example: "https://api.example.com/mcp"
+        headers:
+          type: array
+          description: HTTP headers to include
+          items:
+            $ref: '#/components/schemas/KeyValueInput'
+
+    SseTransport:
       type: object
       required:
         - type
         - url
       properties:
         type:
           type: string
-          enum: [streamable-http, sse]
-          description: Transport protocol type
+          enum: [sse]
+          description: Transport type
           example: "sse"
         url:
           type: string
           format: uri
-          description: Remote server URL
+          description: Server-Sent Events endpoint URL
           example: "https://mcp-fs.example.com/sse"
         headers:
           type: array
@@ -536,7 +574,9 @@ components:
         remotes:
           type: array
           items:
-            $ref: '#/components/schemas/Remote'
+            anyOf:
+              - $ref: '#/components/schemas/StreamableHttpTransport'
+              - $ref: '#/components/schemas/SseTransport'
         _meta:
           type: object
           description: Extension metadata using reverse DNS namespacing