Skip to content

Commit 7f36450

Browse files
jongioCopilot
jongio
and
Copilot
authored
Issue #1085: Prefer AZD pattern modules first, fallback to AVM modules (#1092)
* fix(issue-1085): enforce AVM+AZD-first guidance and add deploy waza Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(agent-runner): restore SDK permission handler compatibility Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(pr-1092): address review feedback and strengthen graders Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(pr-1092): address new copilot review threads - narrow azure-prepare trigger wording to avoid migration overlap\n- make fallback wording provider-scoped in iac-rules\n- update azure-prepare trigger snapshots\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(ci): restore azure-deploy post-deploy guidance - add post-deployment SQL/EF migration step back to azure-deploy workflow\n- restore post-deployment reference link\n- update azure-deploy trigger snapshots\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test: use copilot-sdk approveAll constant Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: restore azure-deploy verify reference link Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(pr-1092): address copilot trigger and emoji feedback Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Trim PR #1092 to AVM-focused changes Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Remove non-AVM scope changes from PR Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: bump skill versions for AVM scope Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent baca73d commit 7f36450

File tree

13 files changed

+216
-7
lines changed

13 files changed

+216
-7
lines changed

‎plugin/skills/azure-deploy/SKILL.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: "Execute Azure deployments for ALREADY-PREPARED applications that h
44
license: MIT
55
metadata:
66
author: Microsoft
7-
version: "1.0.2"
7+
version: "1.0.3"
88
---
99

1010
# Azure Deploy
@@ -86,4 +86,4 @@ Activate this skill when user wants to:
8686
## References
8787

8888
- [Troubleshooting](references/troubleshooting.md) - Common issues and solutions
89-
- [Post-Deployment Steps](references/recipes/azd/post-deployment.md) - SQL + EF Core setup
89+
- [Post-Deployment Steps](references/recipes/azd/post-deployment.md) - SQL + EF Core setup

‎plugin/skills/azure-deploy/references/recipes/bicep/README.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,16 @@ az deployment sub show \
7777
| `mcp_bicep_get_az_resource_type_schema` | Resource schemas |
7878
| `mcp_bicep_list_avm_metadata` | Azure Verified Modules |
7979

80+
## AVM Verification Before Deploy
81+
82+
Before running deployment commands, verify generated templates followed AVM-first module selection:
83+
84+
1. AVM Bicep Pattern Modules (prefer AVM+AZD patterns)
85+
2. AVM Bicep Resource Modules
86+
3. AVM Bicep Utility Modules
87+
88+
If no AVM+AZD pattern module is available, fallback must remain within AVM modules (resource -> utility).
89+
8090
## Cleanup (DESTRUCTIVE)
8191

8292
```bash

‎plugin/skills/azure-prepare/SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: "Prepare Azure apps for deployment (infra Bicep/Terraform, azure.ya
44
license: MIT
55
metadata:
66
author: Microsoft
7-
version: "1.0.1"
7+
version: "1.0.2"
88
---
99

1010
# Azure Prepare

‎plugin/skills/azure-prepare/references/recipes/azcli/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ Azure CLI workflow for imperative Azure deployments.
1717
| Artifact | Research Action |
1818
|----------|-----------------|
1919
| Bicep files | Call `mcp_bicep_get_bicep_best_practices` |
20-
| Bicep modules | Call `mcp_bicep_list_avm_metadata` for Azure Verified Modules |
20+
| Bicep modules | Call `mcp_bicep_list_avm_metadata` and follow [AVM module order](../azd/iac-rules.md#avm-module-selection-order-mandatory) |
2121
| Azure CLI commands | Call `activate_azure_cli_management_tools` |
2222
| Azure best practices | Call `mcp_azure_mcp_get_bestpractices` |
2323

‎plugin/skills/azure-prepare/references/recipes/azd/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ Azure Developer CLI workflow for preparing Azure deployments.
4646
| Azure Functions templates | [Templates](../../services/functions/templates/README.md) |
4747
| Bicep best practices | `mcp_bicep_get_bicep_best_practices` |
4848
| Bicep resource schema | `mcp_bicep_get_az_resource_type_schema` |
49-
| Azure Verified Modules | `mcp_bicep_list_avm_metadata` |
49+
| Azure Verified Modules | `mcp_bicep_list_avm_metadata` + [AVM module order](iac-rules.md#avm-module-selection-order-mandatory) |
5050
| Terraform best practices | `mcp_azure_mcp_azureterraformbestpractices` |
5151
| Dockerfiles | [Docker Guide](docker.md) |
5252

‎plugin/skills/azure-prepare/references/recipes/azd/iac-rules.md‎

Lines changed: 32 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,37 @@
11
# AZD IAC Rules
22

3-
Bicep rules for AZD projects. **Additive** — apply `mcp_bicep_get_bicep_best_practices`, `mcp_bicep_list_avm_metadata`, and `mcp_bicep_get_az_resource_type_schema` first, then these azd-specific rules.
3+
IaC rules for AZD projects. **Additive** — for Bicep, apply `mcp_bicep_get_bicep_best_practices`, `mcp_bicep_list_avm_metadata`, and `mcp_bicep_get_az_resource_type_schema` first; for Terraform, apply `mcp_azure_mcp_azureterraformbestpractices` first; then apply these azd-specific rules.
4+
5+
## AVM Module Selection Order (MANDATORY)
6+
7+
Always prefer modules in provider-specific order:
8+
9+
For **Bicep**:
10+
1. AVM Bicep Pattern Modules (AVM+AZD first when available)
11+
2. AVM Bicep Resource Modules
12+
3. AVM Bicep Utility Modules
13+
14+
For **Terraform**:
15+
1. AVM Terraform Pattern Modules
16+
2. AVM Terraform Resource Modules
17+
3. AVM Terraform Utility Modules
18+
19+
If no pattern module exists for the active provider, default immediately to AVM modules in the same provider order (resource, then utility) instead of using non-AVM modules.
20+
21+
## Retrieval Strategy (Hybrid: azure-documentation MCP + Context7)
22+
23+
- **Primary (authoritative):** Use `mcp_azure_mcp_documentation` (`azure-documentation`) for current Azure guidance and AVM integration documentation.
24+
- **Primary (module catalog):** Use `mcp_bicep_list_avm_metadata` plus official AVM indexes to select concrete modules.
25+
- **Secondary (supplemental):** Use Context7 only for implementation examples when `mcp_azure_mcp_documentation` does not provide enough detail.
26+
27+
## Validation Plan
28+
29+
Before finalizing generated guidance:
30+
31+
1. Verify the selected module path uses the required AVM order above.
32+
2. Verify AVM+AZD pattern modules were checked first, and fallback moved to AVM resource/utility modules when no pattern module exists.
33+
3. Verify Terraform guidance follows pattern -> resource -> utility ordering.
34+
4. Include selected module names and source links in the plan/output for traceability.
435

536
## File Structure
637

‎plugin/skills/azure-prepare/references/recipes/azd/terraform.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -274,6 +274,16 @@ When preparing a new azd+Terraform project:
274274
- Hosting resources: `azd-service-name` (matches azure.yaml services)
275275
4. **Research best practices** - Call `mcp_azure_mcp_azureterraformbestpractices`
276276

277+
## AVM Terraform Module Priority
278+
279+
For Terraform module selection, enforce this order:
280+
281+
1. AVM Terraform Pattern Modules
282+
2. AVM Terraform Resource Modules
283+
3. AVM Terraform Utility Modules
284+
285+
Use `mcp_azure_mcp_documentation` (`azure-documentation`) for current guidance and AVM context first, then use Context7 only as supplemental examples if required.
286+
277287
## Migration from Pure Terraform
278288

279289
Converting existing Terraform project to use azd:

‎plugin/skills/azure-prepare/references/recipes/bicep/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ Standalone Bicep workflow (without AZD).
1717
| Artifact | Research Action |
1818
|----------|-----------------|
1919
| Bicep files | Call `mcp_bicep_get_bicep_best_practices` |
20-
| Bicep modules | Call `mcp_bicep_list_avm_metadata` for Azure Verified Modules |
20+
| Bicep modules | Call `mcp_bicep_list_avm_metadata` and follow [AVM module order](../azd/iac-rules.md#avm-module-selection-order-mandatory) |
2121
| Resource schemas | Use `activate_azure_resource_schema_tools` if needed |
2222

2323
## Generation Steps

‎tests/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -166,6 +166,7 @@ go install github.com/microsoft/waza/cmd/waza@latest
166166

167167
**Committed eval suites** (⬢ customized graders, fixtures, and assertions):
168168
- `azure-prepare` — template selection, recipe composition, plan-first workflow
169+
- `azure-deploy` — deploy routing and AVM+AZD module-priority/fallback guidance
169170

170171
**Auto-generated** (⬡ from SKILL.md frontmatter): all other skills
171172

‎tests/azure-deploy/eval/eval.yaml‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# Waza eval specification for azure-deploy skill
2+
# Focuses on deploy-time AVM guidance and fallback behavior from issue #1085.
3+
name: azure-deploy-eval
4+
description: |
5+
Evaluation suite for the azure-deploy skill.
6+
Tests deployment guidance quality with emphasis on:
7+
- AVM+AZD pattern-module preference
8+
- AVM fallback when no pattern module exists
9+
- deploy-only routing (not prepare/validate)
10+
11+
skill: azure-deploy
12+
version: "1.0"
13+
14+
config:
15+
trials_per_task: 3
16+
timeout_seconds: 420
17+
parallel: false
18+
executor: mock
19+
model: claude-sonnet-4-20250514
20+
21+
metrics:
22+
- name: task_completion
23+
weight: 0.4
24+
threshold: 0.8
25+
description: Did the skill complete a deploy-focused request?
26+
27+
- name: trigger_accuracy
28+
weight: 0.3
29+
threshold: 0.85
30+
description: Does the skill trigger on deploy prompts and avoid prepare-only prompts?
31+
32+
- name: behavior_quality
33+
weight: 0.3
34+
threshold: 0.75
35+
description: Does output preserve AVM+AZD preference and AVM fallback order?
36+
37+
graders:
38+
- type: code
39+
name: has_output
40+
config:
41+
assertions:
42+
- "len(output) > 10"
43+
44+
- type: regex
45+
name: no_runtime_failure
46+
config:
47+
must_not_match:
48+
- "(?i)fatal error|exception occurred|crashed"
49+
50+
tasks:
51+
- "tasks/*.yaml"

0 commit comments

Comments
 (0)