|
| 1 | +# Strapi Release Documentation Analyzer |
| 2 | + |
| 3 | +Automated analysis tool for Strapi releases that generates intelligent documentation update suggestions using Claude AI. |
| 4 | + |
| 5 | +## 📋 Overview |
| 6 | + |
| 7 | +This tool analyzes Strapi release notes, examines each mentioned Pull Request, and uses Claude AI to generate precise, actionable documentation update suggestions. It helps the documentation team quickly identify what needs to be updated after each release. |
| 8 | + |
| 9 | +## ✨ Features |
| 10 | + |
| 11 | +- **Automated PR Analysis**: Fetches and analyzes all PRs mentioned in a release note |
| 12 | +- **Intelligent Categorization**: Automatically categorizes PRs (features, bugs, enhancements) |
| 13 | +- **Smart Filtering**: Excludes chores, tests, and CI/CD changes that don't need documentation |
| 14 | +- **AI-Powered Suggestions**: Uses Claude Sonnet 4.5 to generate specific, actionable documentation updates |
| 15 | +- **Complete Diff Analysis**: Examines actual code changes to understand impact |
| 16 | +- **Ready-to-Use Content**: Provides markdown content that can be directly added to documentation |
| 17 | +- **Priority Assessment**: Automatically determines urgency (high/medium/low) |
| 18 | +- **Comprehensive Reports**: Generates detailed markdown reports organized by category |
| 19 | + |
| 20 | +## 🚀 Installation |
| 21 | + |
| 22 | +### Prerequisites |
| 23 | + |
| 24 | +- Node.js >= 18.0 |
| 25 | +- Yarn (the project uses yarn as package manager) |
| 26 | + |
| 27 | +### Dependencies |
| 28 | + |
| 29 | +The script requires: |
| 30 | +- `@octokit/rest` (GitHub API client) - already in root project |
| 31 | +- `dotenv` (environment variables) - already in root project |
| 32 | +- `@anthropic-ai/sdk` (Claude AI) - needs to be installed |
| 33 | + |
| 34 | +Install the Anthropic SDK dependency: |
| 35 | + |
| 36 | +```bash |
| 37 | +cd scripts/strapi-release-analyzer |
| 38 | +yarn install |
| 39 | +``` |
| 40 | + |
| 41 | +## 🔑 Configuration |
| 42 | + |
| 43 | +### 1. GitHub Token |
| 44 | + |
| 45 | +Create a GitHub Personal Access Token: |
| 46 | +1. Go to https://github.com/settings/tokens |
| 47 | +2. Generate new token (classic) |
| 48 | +3. Select scope: `public_repo` (read access to public repositories) |
| 49 | +4. Copy the token |
| 50 | + |
| 51 | +### 2. Anthropic API Key |
| 52 | + |
| 53 | +Get your Claude API key: |
| 54 | +1. Go to https://console.anthropic.com/settings/keys |
| 55 | +2. Create a new API key |
| 56 | +3. Copy the key |
| 57 | + |
| 58 | +### 3. Environment Variables |
| 59 | + |
| 60 | +Create a `.env` file at the project root: |
| 61 | + |
| 62 | +```bash |
| 63 | +GITHUB_TOKEN=ghp_your_github_token_here |
| 64 | +ANTHROPIC_API_KEY=sk-ant-your_anthropic_key_here |
| 65 | +``` |
| 66 | + |
| 67 | +## 📖 Usage |
| 68 | + |
| 69 | +From the documentation repository root: |
| 70 | + |
| 71 | +```bash |
| 72 | +node scripts/strapi-release-analyzer/index.js <release-url> |
| 73 | +``` |
| 74 | + |
| 75 | +### Example |
| 76 | + |
| 77 | +```bash |
| 78 | +node scripts/strapi-release-analyzer/index.js https://github.com/strapi/strapi/releases/tag/v5.29.0 |
| 79 | +``` |
| 80 | + |
| 81 | +### Output |
| 82 | + |
| 83 | +The script generates a markdown file in `release-analysis/`: |
| 84 | + |
| 85 | +``` |
| 86 | +release-analysis/ |
| 87 | +└── v5.29.0-doc-analysis.md |
| 88 | +``` |
| 89 | + |
| 90 | +## 📊 Report Structure |
| 91 | + |
| 92 | +The generated report includes: |
| 93 | + |
| 94 | +### 1. Summary |
| 95 | +- Total PRs analyzed |
| 96 | +- Breakdown by category (features, bugs, enhancements) |
| 97 | +- Breakdown by priority (high, medium, low) |
| 98 | + |
| 99 | +### 2. Detailed Analysis per PR |
| 100 | +For each analyzed PR, you get: |
| 101 | + |
| 102 | +- **Priority Level**: Visual indicator (🔴 high, 🟡 medium, 🟢 low) |
| 103 | +- **PR Link**: Direct link to the pull request |
| 104 | +- **Affected Areas**: Specific documentation sections impacted |
| 105 | +- **Analysis**: Claude's reasoning about why documentation needs updates |
| 106 | +- **Suggested Changes**: Concrete, actionable suggestions including: |
| 107 | + - File path to update (using current structure: /cms/ and /cloud/) |
| 108 | + - Section to modify |
| 109 | + - Change type (add/update/clarify/add-note) |
| 110 | + - Description of what to change |
| 111 | + - **Ready-to-use markdown content** |
| 112 | + |
| 113 | +**Note**: The analyzer is aware of Strapi's current documentation structure: |
| 114 | +- `/cms/` folder for CMS documentation (replaces old /user-docs and /dev-docs) |
| 115 | +- `/cloud/` folder for Cloud documentation |
| 116 | +- Features in `/cms/features/` |
| 117 | +- API docs in `/cms/api/` |
| 118 | +- Configurations in `/cms/configurations/` |
| 119 | + |
| 120 | +### 3. Notes |
| 121 | +- Methodology information |
| 122 | +- Disclaimer about manual review |
| 123 | +- Usage recommendations |
| 124 | + |
| 125 | +## 🎯 Priority Logic |
| 126 | + |
| 127 | +The AI determines priority based on: |
| 128 | + |
| 129 | +- **🔴 High Priority**: |
| 130 | + - New features |
| 131 | + - Breaking changes |
| 132 | + - API modifications |
| 133 | + - New configuration options |
| 134 | + - Significant behavior changes |
| 135 | + |
| 136 | +- **🟡 Medium Priority**: |
| 137 | + - Important bug fixes affecting workflows |
| 138 | + - UI/UX changes |
| 139 | + - Plugin modifications |
| 140 | + - Internationalization changes |
| 141 | + |
| 142 | +- **🟢 Low Priority**: |
| 143 | + - Minor fixes |
| 144 | + - Cosmetic changes |
| 145 | + - Internal refactoring with minimal user impact |
| 146 | + |
| 147 | +## 💰 Cost Estimation |
| 148 | + |
| 149 | +Using Claude Sonnet 4.5: |
| 150 | +- **Per PR**: ~8,000 tokens on average (varies 2K-30K based on diff size) |
| 151 | +- **Per Release**: ~180,000 tokens (~$1.50 total) |
| 152 | + - Input: ~$0.54 |
| 153 | + - Output: ~$0.75 |
| 154 | + |
| 155 | +The cost is minimal compared to the time saved (estimated 2-3 hours of manual work per release). |
| 156 | + |
| 157 | +## 🔍 What Gets Analyzed |
| 158 | + |
| 159 | +The script examines: |
| 160 | +- ✅ Pull request title and description |
| 161 | +- ✅ Complete diff of all code changes |
| 162 | +- ✅ Files modified (type and location) |
| 163 | +- ✅ PR labels and categories |
| 164 | +- ❌ Automatically excludes: chores, tests, CI/CD changes |
| 165 | + |
| 166 | +## 🛠️ Advanced Usage |
| 167 | + |
| 168 | +### Custom Analysis |
| 169 | + |
| 170 | +You can modify detection rules in the `generateDocSuggestionsWithClaude()` function to: |
| 171 | +- Adjust priority thresholds |
| 172 | +- Add new documentation areas |
| 173 | +- Customize the AI prompt |
| 174 | +- Change output format |
| 175 | + |
| 176 | +### Rate Limiting |
| 177 | + |
| 178 | +The script includes: |
| 179 | +- 1-second delay between API calls |
| 180 | +- Automatic diff truncation for very large PRs (>100KB) |
| 181 | +- Error handling with detailed logging |
| 182 | + |
| 183 | +## ⚠️ Limitations |
| 184 | + |
| 185 | +- AI suggestions require manual review and validation |
| 186 | +- Some internal changes may not need documentation despite being flagged |
| 187 | +- Very large PRs may have truncated diffs |
| 188 | +- API calls respect GitHub and Anthropic rate limits |
| 189 | + |
| 190 | +## 🔮 Future Integration |
| 191 | + |
| 192 | +This tool is designed to be integrated into an automated workflow: |
| 193 | +1. New Strapi release is published |
| 194 | +2. Webhook triggers Slack notification |
| 195 | +3. Script automatically runs and generates report |
| 196 | +4. Report is posted to dedicated Slack channel |
| 197 | +5. Documentation team reviews and acts on suggestions |
| 198 | + |
| 199 | +## 📝 Example Output |
| 200 | + |
| 201 | +```markdown |
| 202 | +### 🔴 PR #24554: feat(upload): fixing ordering issue |
| 203 | + |
| 204 | +**Link:** https://github.com/strapi/strapi/pull/24554 |
| 205 | +**Priority:** HIGH |
| 206 | +**Affected Documentation Areas:** |
| 207 | +- Media Library documentation |
| 208 | +- User Guide - Upload Plugin |
| 209 | + |
| 210 | +**Analysis:** |
| 211 | +This PR fixes a critical ordering issue in the upload functionality that affects |
| 212 | +how users see their media files in the Media Library. Documentation should be |
| 213 | +updated to reflect the correct behavior. |
| 214 | + |
| 215 | +**Suggested Documentation Changes:** |
| 216 | + |
| 217 | +#### 1. UPDATE: Media Library - Sorting and Ordering |
| 218 | + |
| 219 | +**File:** `docs/user-docs/media-library/organizing-assets.md` |
| 220 | + |
| 221 | +**What to do:** Add a note about the fix for ordering consistency and update |
| 222 | +any screenshots showing the file list. |
| 223 | + |
| 224 | +**Suggested content:** |
| 225 | +\`\`\`markdown |
| 226 | +:::note Fix in v5.29.0 |
| 227 | +As of version 5.29.0, the ordering of uploaded files has been fixed to maintain |
| 228 | +consistent display order in the Media Library. Files now appear in the correct |
| 229 | +chronological order based on their upload time. |
| 230 | +::: |
| 231 | +\`\`\` |
| 232 | +``` |
| 233 | + |
| 234 | +## 🤝 Contributing |
| 235 | + |
| 236 | +To improve the analyzer: |
| 237 | +1. Add new detection patterns in `categorizePR()` |
| 238 | +2. Enhance the AI prompt in `generateDocSuggestionsWithClaude()` |
| 239 | +3. Improve report formatting in `generateMarkdownReport()` |
| 240 | +4. Test with multiple releases to validate accuracy |
| 241 | + |
| 242 | +## 📄 License |
| 243 | + |
| 244 | +MIT |
0 commit comments