feat: implement tool name and parameter validation system with eslint integration (#337)

# Fix Claude API Errors with Tool Name Validation

This PR fixes a problem where the Azure DevOps MCP server was getting
error messages from Claude API because some tool names didn't follow the
required format. Claude API is very strict about tool names, they can
only contain letters, numbers, underscores, dots, and dashes, and must
be 64 characters or less. When tool names don't follow these rules,
Claude returns a 400 error and refuses to work.

To solve this, we added validation that checks all tool names before
they get sent to Claude. The validation happens in three places: when
you're writing code (through ESLint), when you build the project, and
when tests run. We tested all 60+ existing tools in the project and they
all pass the validation. The longest tool name is only 40 characters, so
we're well within the limits.

The validation code is organized so there's no duplication, one shared
module handles all the checking logic. We also added comprehensive tests
to make sure the validation works correctly, and wrote documentation to
help developers understand the naming rules. This prevents future API
errors and makes the development process smoother since developers get
immediate feedback when they use invalid tool names.

Example linter error while developing;

<img width="1075" height="121" alt="image"
src="https://github.com/user-attachments/assets/04320b1a-a833-45d8-870e-90ad8a05645d"
/>

Output of validation for current tools;

<img width="667" height="1331" alt="image"
src="https://github.com/user-attachments/assets/5d903b57-efa1-4ecc-ac46-b9fb7dbfcc45"
/>

## GitHub issue number

Fixes #332 

## **Associated Risks**

None

## ✅ **PR Checklist**

- [x] **I have read the [contribution
guidelines](https://github.com/microsoft/azure-devops-mcp/blob/main/CONTRIBUTING.md)**
- [x] **I have read the [code of conduct
guidelines](https://github.com/microsoft/azure-devops-mcp/blob/main/CODE_OF_CONDUCT.md)**
- [x] Title of the pull request is clear and informative.
- [x] 👌 Code hygiene
- [x] 🔭 Telemetry added, updated, or N/A
- [x] 📄 Documentation added, updated, or N/A
- [x] 🛡️ Automated tests added, or N/A

## 🧪 **How did you test it?**

Run `npm run validate-tools` command

Author: polatengin

.github/workflows/build.yml

@@ -42,6 +42,9 @@ jobs:
       - name: Build the project
         run: npm run build
 
+      - name: Validate tool names and parameters
+        run: npm run validate-tools
+
       - name: Run tests
         run: npm test
 

docs/TOOL-NAME-VALIDATION.md

@@ -0,0 +1,226 @@
+# Tool Name Validation Guardrails
+
+This document describes the validation system for Azure DevOps MCP server tool names and parameter names to prevent Claude API validation errors.
+
+## Problem Statement
+
+Claude API has strict requirements for tool names and parameter names:
+
+- Must match pattern: `^[a-zA-Z0-9_.-]{1,64}$`
+- Maximum length: 64 characters
+- Only alphanumeric characters, underscores, dots, and hyphens are allowed
+- No spaces or special characters
+
+Failing to comply results in errors like:
+
+```json
+API Error: 400
+{"type":"error","error":{"type":"invalid_request_error","message":"tools.127.custom.input_schema.properties: Property keys should match pattern '^[a-zA-Z0-9_.-]{1,64}$'"}}
+```
+
+## Guardrails Implemented
+
+### 1. Build-time Validation Script
+
+**Location:** `scripts/build-validate-tools.js`
+
+**Description:** Scans all tool files and validates tool names and parameter names.
+
+**Usage:**
+
+```bash
+npm run validate-tools
+```
+
+**Features:**
+
+- Extracts tool names from `*_TOOLS` constant objects
+- Extracts parameter names from Zod schema definitions
+- Validates against Claude API requirements
+- Shows character counts for each name
+- Warns about parameter names longer than 32 characters (recommendation)
+- Fails build if invalid names are found
+
+### 2. ESLint Rule
+
+**Location:** `eslint-rules/tool-name-lint-rule.js`
+
+**Description:** Custom ESLint rule that validates tool names and parameter names during development.
+
+**Features:**
+
+- Real-time validation in IDE
+- Integrated with existing ESLint configuration
+- Shows errors for invalid names
+- Warnings for long parameter names
+
+**Configuration in `eslint.config.mjs`:**
+
+```javascript
+{
+  files: ["src/tools/*.ts"],
+  plugins: {
+    "custom": {
+      rules: {
+        "validate-tool-names": validateToolNamesRule,
+      },
+    },
+  },
+  rules: {
+    "custom/validate-tool-names": "error",
+  },
+}
+```
+
+### 3. Shared Validation Module
+
+**Location:** `src/shared/tool-validation.ts`
+
+**Functions:**
+
+- `validateToolName(toolName: string)`: Validates a single tool name
+- `validateParameterName(paramName: string)`: Validates a single parameter name
+- `validateName(name: string)`: Core validation function used by both tool and parameter validation
+- `extractToolNames(fileContent: string)`: Extracts tool names from TypeScript files
+- `extractParameterNames(fileContent: string)`: Extracts parameter names from Zod schemas
+
+**Usage:**
+
+```typescript
+import { validateToolName, validateParameterName } from "./shared/tool-validation.js";
+
+const validation = validateToolName("my_tool_name");
+if (!validation.isValid) {
+  console.error(validation.error);
+}
+```
+
+### 4. Test Coverage
+
+**Location:** `test/src/tool-name-validation.test.ts`
+
+**Coverage:** 100% statements, branches, functions, and lines
+
+**Test Categories:**
+
+- **Validation Functions**: Tests for `validateToolName` and `validateParameterName` with valid/invalid inputs
+- **Character Patterns**: Tests for all valid characters (a-z, A-Z, 0-9, \_, ., -) and invalid characters
+- **Length Limits**: Edge cases for 64-character maximum length
+- **Extraction Functions**: Tests for `extractToolNames` and `extractParameterNames` with real-world patterns
+- **Edge Cases**: Empty inputs, malformed content, mixed file content
+
+**Running Tests:**
+
+```bash
+# Run all validation tests
+npm test test/src/tool-name-validation.test.ts
+
+# Run with coverage report
+npm test test/src/tool-name-validation.test.ts -- --coverage
+```
+
+## Architecture Overview
+
+The validation system uses a **consolidated architecture** with shared validation logic:
+
+```bash
+src/shared/tool-validation.ts         # Single source of truth for validation logic
+├── validateName()                    # Core validation function
+├── validateToolName()                # Tool-specific validation
+├── validateParameterName()           # Parameter-specific validation
+├── extractToolNames()                # Extract tools from TypeScript files
+└── extractParameterNames()           # Extract parameters from Zod schemas
+
+scripts/build-validate-tools.js       # Build-time validation (imports shared module)
+eslint-rules/tool-name-lint-rule.js   # ESLint rule (imports shared module)
+test/src/tool-name-validation.test.ts # Comprehensive tests (100% coverage)
+```
+
+**Benefits:**
+
+- **No Code Duplication**: Single validation implementation
+- **Consistent Behavior**: Same logic across build-time and development-time validation
+- **Maintainable**: Changes in one place update all validation systems
+- **Well-Tested**: 100% test coverage ensures reliability
+
+## Best Practices
+
+### Tool Naming
+
+- Use descriptive but concise names
+- Follow the pattern: `{category}_{action}_{object}`
+- Examples: `repo_create_pull_request`, `build_get_status`
+- Maximum 64 characters (current longest is 40 characters)
+
+### Parameter Naming
+
+- Use clear, descriptive names
+- Avoid abbreviations unless commonly understood
+- Keep under 32 characters for readability (recommendation)
+- Examples: `project`, `repositoryId`, `includeDetails`
+
+### Development Workflow
+
+1. Create new tools with descriptive names
+2. Run `npm run validate-tools` to check compliance
+3. Fix any validation errors before committing
+4. ESLint will catch issues in real-time during development
+
+## Validation Rules
+
+### Valid Characters
+
+- Alphanumeric: `a-z`, `A-Z`, `0-9`
+- Underscore: `_`
+- Dot: `.`
+- Hyphen: `-`
+
+### Invalid Characters
+
+- Spaces: ` `
+- Forward slash: `/`
+- Special characters: `@`, `#`, `$`, `%`, etc.
+
+### Length Limits
+
+- Minimum: 1 character
+- Maximum: 64 characters
+- Recommended for parameters: ≤32 characters
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Tool name contains spaces**
+
+   ```text
+   Error: Tool name 'my tool name' contains invalid characters
+   ```
+
+   Solution: Use underscores instead: `my_tool_name`
+
+2. **Tool name too long**
+
+   ```text
+   Error: Tool name 'very_long_descriptive_tool_name_that_exceeds_limit' is 45 characters long, maximum allowed is 64
+   ```
+
+   Solution: Shorten the name: `long_tool_name`
+
+3. **Special characters in name**
+
+   ```text
+   Error: Tool name 'tool/name' contains invalid characters
+   ```
+
+   Solution: Use allowed characters: `tool_name`
+
+### Running Validation
+
+```bash
+# Validate all tools
+npm run validate-tools
+
+# Run ESLint on tool files
+npx eslint src/tools/*.ts
+```

eslint-rules/tool-name-lint-rule.js

@@ -0,0 +1,91 @@
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT License.
+
+import { validateName } from "../dist/shared/tool-validation.js";
+
+/**
+ * Custom ESLint rule to validate Azure DevOps MCP tool names and parameter names
+ */
+export default {
+  meta: {
+    type: "problem",
+    docs: {
+      description: "Validate tool names and parameter names conform to Claude API requirements",
+      category: "Possible Errors",
+      recommended: true,
+    },
+    fixable: null,
+    schema: [],
+    messages: {
+      invalidToolName: "Tool name '{{name}}' is invalid: {{reason}}",
+      invalidParameterName: "Parameter name '{{name}}' is invalid: {{reason}}",
+      longParameterName: "Parameter name '{{name}}' is {{length}} characters long, consider shortening for better readability",
+    },
+  },
+
+  create(context) {
+    return {
+      // Check tool constant definitions like: toolName: "actual_tool_name"
+      Property(node) {
+        if (node.value && node.value.type === "Literal" && typeof node.value.value === "string" && node.parent && node.parent.type === "ObjectExpression") {
+          const parentObject = node.parent.parent;
+
+          // Check if this is a tool constants object (contains "_TOOLS")
+          if (parentObject && parentObject.type === "VariableDeclarator" && parentObject.id && parentObject.id.name && parentObject.id.name.includes("_TOOLS")) {
+            const toolName = node.value.value;
+            const validation = validateName(toolName);
+
+            if (!validation.isValid) {
+              context.report({
+                node: node.value,
+                messageId: "invalidToolName",
+                data: {
+                  name: toolName,
+                  reason: validation.reason,
+                },
+              });
+            }
+          }
+        }
+      },
+
+      // Check parameter names in function calls like: paramName: z.string()
+      CallExpression(node) {
+        if (
+          node.callee &&
+          node.callee.type === "MemberExpression" &&
+          node.callee.object &&
+          node.callee.object.name === "z" &&
+          node.parent &&
+          node.parent.type === "Property" &&
+          node.parent.key &&
+          node.parent.key.type === "Identifier"
+        ) {
+          const paramName = node.parent.key.name;
+          const validation = validateName(paramName);
+
+          if (!validation.isValid) {
+            context.report({
+              node: node.parent.key,
+              messageId: "invalidParameterName",
+              data: {
+                name: paramName,
+                reason: validation.reason,
+              },
+            });
+          } else if (paramName.length > 32) {
+            // Warning for long parameter names
+            context.report({
+              node: node.parent.key,
+              messageId: "longParameterName",
+              data: {
+                name: paramName,
+                length: paramName.length,
+              },
+            });
+          }
+        }
+      },
+    };
+  },
+};

eslint.config.mjs

@@ -1,6 +1,7 @@
 import pluginHeader from "eslint-plugin-header";
 import tseslint from "typescript-eslint";
 import eslintConfigPrettier from "eslint-config-prettier";
+import validateToolNamesRule from "./eslint-rules/tool-name-lint-rule.js";
 
 pluginHeader.rules.header.meta.schema = false; // workaround for https://github.com/Stuk/eslint-plugin-header/issues/57
 
@@ -27,6 +28,21 @@ export default tseslint.config(
     },
   },
 
+  // Tool name validation for MCP tools
+  {
+    files: ["src/tools/*.ts"],
+    plugins: {
+      custom: {
+        rules: {
+          "validate-tool-names": validateToolNamesRule,
+        },
+      },
+    },
+    rules: {
+      "custom/validate-tool-names": "error",
+    },
+  },
+
   // Prettier integration (must be last)
   eslintConfigPrettier
 );