[MCP] Modular inputs + Enhanced Quality Tests (#28460)

Co-authored-by: Copilot <[email protected]>

Author: notyashhh

.gitignore

@@ -233,6 +233,7 @@ launchSettings.json
 /tools/Modules/tmp
 /tools/Az/Az.psm1
 /tools/AzPreview/AzPreview.psm1
+/tools/Mcp/.logs
 /Azure.PowerShell.sln
 
 # Added due to scan

tools/Mcp/README.md

@@ -1,42 +1,64 @@
 # Azure PowerShell Codegen MCP Server
 
-A Model Context Protocol (MCP) server that provides tools for generating and managing Azure PowerShell modules using AutoRest. This server helps automate common tasks in the Azure PowerShell code generation process, including handling polymorphism, model directives, and code generation.
+A Model Context Protocol (MCP) server that provides tools for generating and managing Azure PowerShell modules using AutoRest. It now also orchestrates help‑driven example generation, CRUD test scaffolding, and an opinionated partner workflow to keep outputs deterministic and consistent.
 
 ## Overview
 
 This MCP server is designed to work with Azure PowerShell module development workflows. It provides specialized tools for:
 
-- **AutoRest Code Generation**: Generate PowerShell modules from OpenAPI specifications
+- **Module Scaffolding**: Interactive selection of service → provider → API version and creation of the `<ModuleName>.Autorest` structure
+- **AutoRest Code Generation**: Generate PowerShell modules from OpenAPI specifications (reset/generate/build sequence)
+- **Example Generation**: Create example scripts from swagger example JSON while filtering strictly to parameters documented in help markdown
+- **Test Generation**: Produce per‑resource CRUD test files (idempotent, includes negative test) using the same help‑driven parameter filtering
+- **Help‑Driven Parameter Filtering**: Only parameters present in the generated help (`/src/<Module>/help/*.md`) are allowed in examples/tests
 - **Model Management**: Handle model directives like `no-inline` and `model-cmdlet`
-- **Polymorphism Support**: Automatically detect and configure polymorphic types
-- **YAML Configuration**: Parse and manipulate AutoRest configuration files
+- **Polymorphism Support**: Automatically detect and configure parent/child discriminator relationships
+- **YAML Configuration Utilities**: Parse and manipulate AutoRest configuration blocks
+- **Partner Workflow Prompt**: A single prompt that encodes the end‑to‑end deterministic workflow
 
 ## Features
 
 ### Available Tools
 
-1. **generate-autorest**
-   - Generates PowerShell code using AutoRest
-   - Parameters: `workingDirectory` (absolute path to README.md)
-
-2. **no-inline**
-   - Converts flattened models to non-inline parameters
-   - Parameters: `modelNames` (array of model names to make non-inline)
-   - Useful for complex nested models that shouldn't be flattened
-
-3. **model-cmdlet**
-   - Creates `New-` cmdlets for specified models
-   - Parameters: `modelNames` (array of model names)
-   - Generates cmdlets with naming pattern: `New-Az{SubjectPrefix}{ModelName}Object`
-
-4. **polymorphism**
-   - Handles polymorphic type detection and configuration
-   - Parameters: `workingDirectory` (absolute path to README.md)
-   - Automatically identifies parent-child type relationships
+1. **setup-module-structure**
+   - Interactive service → provider → API version selection and module name capture
+   - Scaffolds `src/<Module>/<Module>.Autorest/` plus initial `README.md`
+   - Output placeholder `{0}` = module name
+
+2. **generate-autorest**
+   - Executes Autorest reset, generate, and PowerShell build steps within the given working directory
+   - Parameters: `workingDirectory` (absolute path to the Autorest folder containing README.md)
+   - Output placeholder `{0}` = working directory
+
+3. **create-example**
+   - Downloads swagger example JSON, filters parameters to those documented in help markdown (`/src/<Module>/help/<Cmdlet>.md`), and writes example scripts under `examples/`
+   - Parameters: `workingDirectory`
+   - Output placeholders: `{0}` = harvested specs path, `{1}` = examples dir, `{2}` = reference ideal example dirs
+
+4. **create-test**
+   - Generates new `<ResourceName>.Crud.Tests.ps1` files (does not modify stubs) with Create/Get/List/Update/Delete/Negative blocks, using help‑filtered parameters
+   - Parameters: `workingDirectory`
+   - Output placeholders: `{0}` = harvested specs path, `{1}` = test dir, `{2}` = reference ideal test dirs
+
+5. **polymorphism**
+   - Detects discriminator parents and child model names to aid directive insertion
+   - Parameters: `workingDirectory`
+   - Output placeholders: `{0}` = parents, `{1}` = children, `{2}` = working directory
+
+6. **no-inline**
+   - Lists models to be marked `no-inline` (caller inserts directive into README Autorest YAML)
+   - Parameters: `modelNames` (array)
+   - Output `{0}` = comma-separated model list
+
+7. **model-cmdlet**
+   - Lists models for which `New-` object construction cmdlets should be added via directives
+   - Parameters: `modelNames` (array)
+   - Output `{0}` = comma-separated model list
 
 ### Available Prompts
 
-- **create-greeting**: Generate customized greeting messages (example prompt)
+- **partner-module-workflow**: Canonical end‑to‑end instruction set (module structure → generation → examples → tests → regeneration)
+- **create-greeting**: Sample/demo greeting prompt
 
 ## Installation
 

tools/Mcp/src/CodegenServer.ts

@@ -1,14 +1,16 @@
 import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
 import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
 import { z } from "zod";
-import { responseSchema, toolParameterSchema, toolSchema, promptSchema } from "./types.js";
+import { responseSchema, toolParameterSchema, toolSchema, promptSchema, resourceSchema } from "./types.js";
 import { ToolsService } from "./services/toolsService.js";
 import { PromptsService } from "./services/promptsService.js";
+import { ResourcesService } from "./services/resourcesService.js";
 import { readFileSync } from "fs";
 import path from "path";
 import { fileURLToPath } from "url";
 import { RequestOptions } from "https";
 import { ElicitRequest, ElicitResult } from "@modelcontextprotocol/sdk/types.js";
+import { logger } from "./services/logger.js";
 
 const __dirname = path.dirname(fileURLToPath(import.meta.url));
 const srcPath = path.resolve(__dirname, "..", "src");
@@ -37,6 +39,7 @@ export class CodegenServer {
         this.initResponses();
         this.initTools();
         this.initPrompts();
+        this.initResources();
     }
 
     // dummy method for sending sampling request
@@ -74,6 +77,9 @@ export class CodegenServer {
         await this._mcp.connect(transport);
     }
 
+    public getResponseTemplate(name: string): string | undefined {
+        return this._responses.get(name);
+    }
 
     initTools() {
         const toolsService = ToolsService.getInstance().setServer(this);
@@ -100,7 +106,44 @@ export class CodegenServer {
                 schema.name,
                 schema.description,
                 parameter,
-                (args: any) => callback(args)
+                async (args: any) => {
+                    const correlationId = `${schema.name}-${Date.now()}-${Math.random().toString(16).slice(2,8)}`;
+                    logger.debug('Prompt started', { prompt: schema.name, correlationId });
+                    try {
+                        const result = await callback(args);
+                        logger.info('Prompt completed', { prompt: schema.name, correlationId });
+                        return result;
+                    } catch (err: any) {
+                        logger.error('Prompt failed', { prompt: schema.name, correlationId }, err);
+                        throw err;
+                    }
+                }
+            );
+        }
+    }
+
+    initResources() {
+        const resourcesService = ResourcesService.getInstance().setServer(this);
+        const resourcesSchemas = (specs.resources || []) as resourceSchema[];
+        for (const schema of resourcesSchemas) {
+            const parameter = resourcesService.createResourceParametersFromSchema(schema.parameters || []);
+            const callback = resourcesService.getResources(schema.callbackName, this._responses.get(schema.name));
+            this._mcp.resource(
+                schema.name,
+                schema.description,
+                parameter,
+                async (args: any) => {
+                    const correlationId = `${schema.name}-${Date.now()}-${Math.random().toString(16).slice(2,8)}`;
+                    logger.debug('Resource requested', { resource: schema.name, correlationId });
+                    try {
+                        const result = await callback(args);
+                        logger.info('Resource provided', { resource: schema.name, correlationId });
+                        return result;
+                    } catch (err: any) {
+                        logger.error('Resource failed', { resource: schema.name, correlationId }, err);
+                        throw err;
+                    }
+                }
             );
         }
     }
@@ -110,14 +153,16 @@ export class CodegenServer {
             let text = response.text;
             if (text.startsWith("@file:")) {
                 const relPath = text.replace("@file:", "");
-                const absPath = path.join(srcPath, "specs", relPath);
+                const absPath = path.join(srcPath, relPath);
                 try {
                     text = readFileSync(absPath, "utf-8");
-                } catch (e) {
-                    console.error(`Failed to load prompt file ${absPath}:`, e);
+                } catch (e: any) {
+                    logger.error(`Failed to load prompt file`, { absPath }, e as Error);
                 }
             }
             this._responses.set(response.name, text);
         });
     }
+
+
 }

tools/Mcp/src/assets/autorest-readme-template.md

@@ -0,0 +1,63 @@
+<!-- region Generated -->
+# Az.{moduleName}
+This directory contains the PowerShell module for the {moduleName} service.
+
+---
+## Info
+- Modifiable: yes
+- Generated: all
+- Committed: yes
+- Packaged: yes
+
+---
+## Detail
+This module was primarily generated via [AutoRest](https://github.com/Azure/autorest) using the [PowerShell](https://github.com/Azure/autorest.powershell) extension.
+
+## Module Requirements
+- [Az.Accounts module](https://www.powershellgallery.com/packages/Az.Accounts/), version 2.7.5 or greater
+
+## Authentication
+AutoRest does not generate authentication code for the module. Authentication is handled via Az.Accounts by altering the HTTP payload before it is sent.
+
+## Development
+For information on how to develop for `Az.{moduleName}`, see [how-to.md](how-to.md).
+<!-- endregion -->
+
+---
+### AutoRest Configuration 
+> see https://aka.ms/autorest 
+
+```yaml 
+
+commit: {commitId}
+
+require: 
+  - $(this-folder)/../../readme.azure.noprofile.md 
+  - $(repo)/specification/{serviceSpecs}/readme.md 
+
+try-require:  
+  - $(repo)/specification/{serviceSpecs}/readme.powershell.md 
+
+input-file:
+  - $(repo)/{swaggerFileSpecs}
+
+module-version: 0.1.0 
+
+title: {moduleName}
+service-name: {moduleName}
+subject-prefix: $(service-name) 
+
+directive: 
+
+  - where: 
+      variant: ^(Create|Update)(?!.*?(Expanded|JsonFilePath|JsonString)) 
+    remove: true 
+
+  - where: 
+      variant: ^CreateViaIdentity$|^CreateViaIdentityExpanded$ 
+    remove: true 
+
+  - where: 
+      verb: Set 
+    remove: true 
+```

tools/Mcp/src/assets/example-instructions.md

@@ -0,0 +1,43 @@
+## LLM Example Generation Directions
+
+You have just called tool `create-example` for a freshly generated module.
+
+Inputs:
+- `{0}` = source swagger example JSON directory (read only)
+- `{1}` = target examples directory (write here only)
+- `{2}` = reference example dirs (style cues; may be empty)
+- helpDir = parentOf({1}) with `.Autorest` removed + `/help` (read only)
+
+Goal: Produce minimal, runnable PowerShell example scripts for each relevant cmdlet using ONLY parameters documented in help.
+
+Algorithm (repeat per cmdlet needed):
+1. Open `helpDir/<CmdletName>.md`.
+2. Collect allowed params = (a) params in first syntax line(s) in code fences + (b) every `### -ParamName` heading. Exclude `CommonParameters`.
+3. For each swagger JSON in `{0}` referencing this cmdlet, map its fields to allowed params; drop non‑allowed silently.
+4. Order parameters: required (in the order of the first syntax signature) then optional alphabetical.
+5. Build one minimal example. Add a second variant ONLY if it demonstrates distinct optional parameters.
+
+Rules:
+* Never invent or rename parameters; casing must match help.
+* Value selection precedence (per allowed parameter):
+	1. If the swagger example JSON (source `{0}`) contains a concrete value for that parameter (after mapping), use that value directly.
+	2. If the swagger value is obviously redacted (e.g. `"string"`, `"<string>"`, `"XXXX"`, empty, or null) then fall back to a stable placeholder instead of using the dummy.
+	3. Otherwise (no concrete usable value) use a stable placeholder: `<ResourceGroupName>`, `<Location>`, `<SubscriptionId>`, `<Name>`, etc.
+* Do not substitute placeholders where a good swagger value exists.
+* If no allowed params remain after filtering, create/leave an empty file or a single comment line.
+* Do not copy help prose; output only script lines (and brief inline comments if helpful).
+* Mirror formatting style hints (indentation, spacing) from reference dirs `{2}` without copying their literal values.
+
+Output Handling:
+- Modify/create files ONLY under `{1}`; no other directories.
+- Preserve existing example files, updating parameter sets/order as needed.
+
+Quick Validation Checklist (stop if any fail):
+1. All parameters exist in help.
+2. Required parameters present & ordered first.
+3. No swagger‑only or duplicate parameters.
+4. Placeholders consistent.
+5. No redundant variant scripts.
+
+Produce the final example script contents now; do not restate these instructions.
+