Skip to content

Commit dc00825

Browse files
authored
feat: Add comprehensive QA testing framework and multi-agent guidance system (#218)
* docs: add comprehensive PRD for deployment architecture refactor - Complete design decisions with rationale - 5 implementation phases with detailed tasks - BDD acceptance criteria for all features - Comprehensive testing strategy - Risk analysis and mitigation - Timeline with Gantt chart - Success metrics and KPIs Addresses packaging issues, version management, and testing gaps. Establishes dual distribution (NuGet + GitHub releases). Implements changelog-driven versioning with Ionide.KeepAChangelog. * feat: add comprehensive QA testing framework and multi-agent guidance system ## QA Testing Framework ### Claude Code Skill - Created .claude/skills/qa-tester/ with F# automation scripts - smoke-test.fsx: Quick 2-minute validation (Build → Test → Package) - regression-test.fsx: Full 10-15 minute regression suite - validate-packages.fsx: NuGet package structure validation - All scripts use Spectre.Console for rich terminal output - Cross-platform (F# via dotnet fsi) ### Phase 1 Test Plan - Comprehensive test plan for Phase 1 completion (PR #214) - 19 test cases (11 critical, 7 high, 1 medium) - Implementation analysis with deviations documented - Test execution procedures and scripts - docs/content/contributing/qa/phase-1-test-plan.md ## Cross-Agent Guidance System ### .agents/ Directory (New Pattern) - Created .agents/ for specialized, domain-specific guidance - .agents/qa-testing.md: Comprehensive QA guidance for all AI agents - Pre-commit and PR verification checklists - Test plan and bug report templates - 5 testing playbooks (regression, feature, build, package, PR) - BDD (Reqnroll) and unit testing (TUnit) guides - Coverage requirements and best practices - .agents/README.md: Navigation and contribution guidelines ### Multi-Agent Pointer Files Created configuration files for all major AI coding assistants: - .cursorrules: Cursor AI pointer and quick reference - .github/copilot-instructions.md: GitHub Copilot instructions - .windsurf/rules/morphir.md: Windsurf AI rules and workflows - .windsurf/README.md: Windsurf configuration guide - .idea/ai-assistant-rules.md: JetBrains AI Assistant rules - .idea/README.md: JetBrains setup and integration guide All pointer files: - Direct to AGENTS.md as primary source - Link to .agents/ for specialized topics - Provide quick reference and tool-specific tips - Follow tool-specific standards and conventions ### AGENTS.md Enhancements - Added Quick Navigation section at top - Links to .agents/, CLAUDE.md, docs/ - Visible immediately after header - Added Section 18: Specialized Guidance - Details available topics in .agents/ - Links to tool-specific guidance - Lists future topics - Added Section 19: Resources and References - Primary documentation links - Testing resources - Morphir resources - Standards and tools ## Benefits ### Discoverability - All major AI tools (Cursor, Copilot, Windsurf, JetBrains AI, Claude Code) can discover guidance - Clear navigation: Tool-specific → AGENTS.md → .agents/ - No orphaned documentation ### QA Automation - F# test scripts provide consistent, repeatable testing - Smoke test catches regressions quickly - Regression suite validates complete functionality - Package validation ensures correct structure ### Cross-Agent Compatibility - Single source of truth (AGENTS.md) - Specialized topics work with any AI agent - Tool-specific optimizations when available ## Standards Compliance - AGENTS.md Standard: Fully compliant (https://agents.md) - Cursor: .cursorrules follows conventions - GitHub Copilot: .github/copilot-instructions.md per GitHub docs - Windsurf: .windsurf/rules/*.md per Windsurf docs - JetBrains: Settings → AI Assistant → Rules per JetBrains docs ## Related Issues - Closes #209 (Phase 1 verification and test plan) - Filed #217 (Deferred helper classes from Phase 1) 🤖 Generated with [Claude Code](https://claude.com/claude-code) * docs: add dotnet tool restore to setup instructions and F# script troubleshooting ## Setup Instructions Enhanced ### AGENTS.md Section 4 - Added First-time Setup subsection - Documents dotnet tool restore, dotnet restore, dotnet husky install - Updated Commands to reference both direct dotnet and ./build.sh commands ### README.md - Added first-time setup command in Basic Commands section - Shows dotnet tool restore before ./build.sh --target Restore - Ensures tools are available before building ### F# Script Troubleshooting - Added troubleshooting section to .claude/skills/qa-tester/README.md - Documents 3 options for resolving package errors - Special notes for Claude Code limitations - Clarifies F# script NuGet package references work in most environments ## Rationale F# scripts use NuGet package references (#r "nuget: Spectre.Console") which require packages to be cached locally. Running 'dotnet tool restore' ensures tools like Husky.Net and others are available, and 'dotnet restore' caches NuGet packages for F# Interactive to use. This is especially important for: - First-time contributors setting up the repo - CI/CD environments - Claude Code skills that may have package installation limitations Fixes potential issues where F# scripts fail with missing package errors.
1 parent 331e327 commit dc00825

File tree

18 files changed

+6461
-7
lines changed

18 files changed

+6461
-7
lines changed

.agents/README.md

Lines changed: 193 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,193 @@
1+
# AI Coding Agent Resources
2+
3+
This directory contains guidance and resources for AI coding agents working on morphir-dotnet.
4+
5+
## Purpose
6+
7+
While [AGENTS.md](../AGENTS.md) in the project root provides comprehensive guidance for all agents, this directory contains:
8+
- **Specialized topic guides** for specific domains (QA, deployment, etc.)
9+
- **Cross-agent compatible resources** that work with any AI coding assistant
10+
- **Reusable templates and scripts** for common tasks
11+
- **Agent-agnostic documentation** that doesn't depend on specific tool features
12+
13+
## Structure
14+
15+
```
16+
.agents/
17+
├── README.md # This file
18+
├── qa-testing.md # QA testing guidance (all agents)
19+
└── [future topics] # Additional domain-specific guides
20+
```
21+
22+
## For AI Coding Agents
23+
24+
### Primary Guidance
25+
Always start with [AGENTS.md](../AGENTS.md) - it's the authoritative source for:
26+
- Project overview and architecture
27+
- Coding conventions and standards
28+
- Testing strategy (TDD)
29+
- Build and deployment procedures
30+
- Decision-making framework
31+
32+
### Specialized Guidance
33+
This directory provides deep-dive guidance for specific areas:
34+
35+
| Topic | File | When to Use |
36+
|-------|------|-------------|
37+
| QA Testing | [qa-testing.md](qa-testing.md) | Creating test plans, running tests, reporting bugs |
38+
| [Future] Deployment | deployment.md | Publishing packages, releases, CI/CD |
39+
| [Future] Documentation | documentation.md | Writing docs, ADRs, PRDs |
40+
41+
### Claude Code Users
42+
If you're using Claude Code, you also have access to:
43+
- **Claude Skills**: [.claude/skills/](../.claude/skills/) - Specialized prompts and scripts
44+
- [qa-tester](../.claude/skills/qa-tester/) - QA testing skill with F# scripts
45+
46+
See [CLAUDE.md](../CLAUDE.md) for Claude Code-specific guidance.
47+
48+
## For Human Contributors
49+
50+
This directory helps you understand:
51+
- What guidance AI agents are following
52+
- What templates and processes are standardized
53+
- How to contribute agent-friendly documentation
54+
55+
### Adding New Agent Guidance
56+
57+
When adding new specialized guidance:
58+
59+
1. **Create focused topic file**: `{topic}.md` in this directory
60+
2. **Use agent-agnostic language**: Avoid tool-specific features
61+
3. **Provide templates**: Include reusable templates and checklists
62+
4. **Link from AGENTS.md**: Add reference in main AGENTS.md
63+
5. **Test with multiple agents**: Verify guidance works across tools
64+
65+
### Template for New Guidance Files
66+
67+
```markdown
68+
# [Topic] Guidance for AI Coding Agents
69+
70+
**Audience**: All AI coding agents (Claude Code, GitHub Copilot, Cursor, etc.)
71+
**Purpose**: [Brief description]
72+
73+
## Quick Reference
74+
75+
[Table of common tasks and actions]
76+
77+
## [Section 1]
78+
79+
[Detailed guidance]
80+
81+
## Templates
82+
83+
[Reusable templates]
84+
85+
## Scripts
86+
87+
[References to scripts]
88+
89+
## Best Practices
90+
91+
[Do's and don'ts]
92+
93+
## Agent-Specific Notes
94+
95+
[Any tool-specific variations]
96+
97+
## Resources
98+
99+
[Links to related docs]
100+
101+
## Updates
102+
103+
**Last Updated**: YYYY-MM-DD
104+
**Maintained By**: [Maintainers]
105+
```
106+
107+
## Relationship to Other Documentation
108+
109+
```
110+
morphir-dotnet/
111+
├── AGENTS.md # Primary agent guidance (START HERE)
112+
├── CLAUDE.md # Claude Code-specific guidance
113+
├── README.md # Project README
114+
├── .agents/ # Specialized agent guidance (THIS DIRECTORY)
115+
│ ├── README.md # This file
116+
│ └── qa-testing.md # QA testing guide
117+
├── .claude/ # Claude Code-specific resources
118+
│ └── skills/ # Claude skills
119+
│ └── qa-tester/ # QA skill with F# scripts
120+
└── docs/ # User-facing documentation
121+
├── content/
122+
└── spec/
123+
```
124+
125+
**Navigation**:
126+
- New to project? → Read `AGENTS.md`
127+
- Need specialized guidance? → Check `.agents/{topic}.md`
128+
- Using Claude Code? → See `CLAUDE.md` and `.claude/skills/`
129+
- Writing docs? → See `docs/`
130+
131+
## Available Guidance
132+
133+
### QA Testing ([qa-testing.md](qa-testing.md))
134+
135+
Comprehensive QA testing guidance including:
136+
- Pre-commit checklist
137+
- PR verification checklist
138+
- Test plan template
139+
- Bug report template
140+
- Regression testing playbook
141+
- Package validation playbook
142+
- Test scripts (F# and C#)
143+
- Coverage requirements
144+
- BDD testing with Reqnroll
145+
- Unit testing with TUnit
146+
147+
**When to use**: Anytime you're creating test plans, running tests, or reporting issues.
148+
149+
### [Future] Deployment Guidance
150+
151+
Coming soon: Guidance for:
152+
- Publishing packages to NuGet
153+
- Creating GitHub releases
154+
- Managing versions
155+
- CI/CD workflows
156+
- Deployment checklists
157+
158+
### [Future] Documentation Guidance
159+
160+
Coming soon: Guidance for:
161+
- Writing documentation
162+
- Creating ADRs (Architecture Decision Records)
163+
- Writing PRDs (Product Requirements Documents)
164+
- API documentation standards
165+
- Examples and tutorials
166+
167+
## Contributing
168+
169+
To improve agent guidance:
170+
171+
1. **Identify gaps**: What tasks are agents struggling with?
172+
2. **Create focused guides**: One topic per file
173+
3. **Test with multiple agents**: Ensure cross-compatibility
174+
4. **Provide examples**: Show, don't just tell
175+
5. **Keep updated**: Revise as project evolves
176+
177+
## Feedback
178+
179+
Found issues with agent guidance?
180+
- File an issue: https://github.com/finos/morphir-dotnet/issues
181+
- Tag with: `agent-guidance`, `documentation`
182+
183+
## Version History
184+
185+
- **2025-12-18**: Initial creation with QA testing guidance
186+
- [Future updates]
187+
188+
## See Also
189+
190+
- [AGENTS.md](../AGENTS.md) - Main agent guidance
191+
- [CLAUDE.md](../CLAUDE.md) - Claude Code guidance
192+
- [Contributing Guide](../docs/content/contributing/) - For all contributors
193+
- [FINOS Code of Conduct](https://www.finos.org/code-of-conduct)

0 commit comments

Comments
 (0)