|
1 | 1 | # AI-powered tools |
2 | 2 |
|
3 | | -A CLI tool for using AI to edit documentation according to defined prompts. |
| 3 | +The ai-tools subject provides CLI tools for using AI to edit and refine documentation based on prompt-driven guidelines. It integrates with GitHub Models API to apply automated improvements to content files. |
4 | 4 |
|
5 | | -This tool refines content files using AI based on an (extensible) set of prompt-driven guidelines. The default is versioning refinement. In the future we might add: scannability, readability, style, technical accuracy. |
| 5 | +## Purpose & Scope |
6 | 6 |
|
7 | | -This script calls the [Models API](https://docs.github.com/en/rest/models/inference?apiVersion=2022-11-28#run-an-inference-request). It requires a personal access token with Models scopes in your `.env` file. |
| 7 | +This subject is responsible for: |
| 8 | +- AI-powered content refinement (versioning, intro, etc.) |
| 9 | +- Prompt-driven content editing with LLMs |
| 10 | +- Integration with GitHub Models API |
| 11 | +- Copilot Spaces export and conversion to prompts |
| 12 | +- Automated content quality improvements |
| 13 | +- Extensible prompt system for different refinement types |
8 | 14 |
|
9 | | -## Usage |
| 15 | +Current refinements: versioning, intro. Future: scannability, readability, style, technical accuracy. |
10 | 16 |
|
11 | | -```sh |
12 | | -<<<<<<< HEAD:src/ai-editors/README.md |
13 | | -tsx src/ai-editors/scripts/ai-edit.ts --editor <type> --response <type> --files <file1.md> |
14 | | -||||||| 5ae4ec0f5cb:src/ai-editors/README.md |
15 | | -tsx src/ai-editors/scripts/ai-edit.js --editor <type> --response <type> --files <file1.md> |
16 | | -======= |
| 17 | +## Architecture & Key Assets |
| 18 | + |
| 19 | +### Key capabilities and their locations |
| 20 | + |
| 21 | +- `scripts/ai-tools.ts` - Main CLI tool for running AI refinements |
| 22 | +- `lib/call-models-api.ts` - Client for GitHub Models API inference |
| 23 | +- `lib/prompt-utils.ts` - Loads prompts and executes refinements |
| 24 | +- `prompts/*.md` - Prompt templates for different refinement types |
| 25 | + |
| 26 | +## Setup & Usage |
| 27 | + |
| 28 | +### Requirements |
| 29 | + |
| 30 | +Add GitHub token with Models scopes to `.env`: |
| 31 | + |
| 32 | +```bash |
| 33 | +GITHUB_TOKEN=ghp_your_token_here |
| 34 | +``` |
| 35 | + |
| 36 | +### Running refinements |
| 37 | + |
| 38 | +```bash |
17 | 39 | # Direct command |
18 | | -tsx src/ai-tools/scripts/ai-tools.ts --refine <type> --files <file1.md> |
| 40 | +tsx src/ai-tools/scripts/ai-tools.ts --refine versioning --files content/path/to/file.md |
19 | 41 |
|
20 | | -# Or via npm script |
21 | | -npm run ai-tools -- --refine <type> --files <file1.md> |
22 | | ->>>>>>> origin/main:src/ai-tools/README.md |
| 42 | +# Via npm script |
| 43 | +npm run ai-tools -- --refine versioning --files content/path/to/file.md |
23 | 44 | ``` |
24 | 45 |
|
25 | | -* `--files, -f`: One or more content file paths to process (required). |
26 | | -* `--refine, -r`: Specify one or more refinement types (default: `versioning`). |
| 46 | +### Options |
27 | 47 |
|
28 | | -**Examples:** |
| 48 | +- `--files, -f`: One or more content file paths (required) |
| 49 | +- `--refine, -r`: Refinement type(s) - `versioning`, `intro` (default: `versioning`) |
| 50 | +- `--write, -w`: Write changes to files (default: false, shows diff only) |
| 51 | +- `--verbose, -v`: Verbose output for debugging |
| 52 | +- `--space, -s`: Use Copilot Space as prompt |
| 53 | +- `--exportSpace`: Export Copilot Space to prompt file |
29 | 54 |
|
30 | | -```sh |
31 | | -<<<<<<< HEAD:src/ai-editors/README.md |
32 | | -tsx src/ai-editors/scripts/ai-edit.ts --files content/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo.md --editor versioning --response list |
33 | | -||||||| 5ae4ec0f5cb:src/ai-editors/README.md |
34 | | -tsx src/ai-editors/scripts/ai-edit.js --files content/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo.md --editor versioning --response list |
35 | | -======= |
36 | | -# Direct command |
37 | | -tsx src/ai-tools/scripts/ai-tools.ts --files content/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo.md --refine versioning |
| 55 | +### Examples |
38 | 56 |
|
39 | | -# Via npm script |
40 | | -npm run ai-tools -- --files content/copilot/tutorials/coding-agent/get-the-best-results.md --refine intro |
41 | | ->>>>>>> origin/main:src/ai-tools/README.md |
| 57 | +Refine versioning in a file: |
| 58 | +```bash |
| 59 | +npm run ai-tools -- --files content/copilot/tutorials/coding-agent/get-the-best-results.md --refine versioning |
42 | 60 | ``` |
43 | 61 |
|
44 | | -## Requirements |
| 62 | +Refine intro: |
| 63 | +```bash |
| 64 | +npm run ai-tools -- --files content/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo.md --refine intro |
| 65 | +``` |
| 66 | + |
| 67 | +Multiple files: |
| 68 | +```bash |
| 69 | +npm run ai-tools -- --files file1.md file2.md file3.md --refine versioning |
| 70 | +``` |
| 71 | + |
| 72 | +Write changes (not just preview): |
| 73 | +```bash |
| 74 | +npm run ai-tools -- --files content/path/to/file.md --refine versioning --write |
| 75 | +``` |
| 76 | + |
| 77 | +## Data & External Dependencies |
| 78 | + |
| 79 | +### Data inputs |
| 80 | +- Content markdown files with frontmatter |
| 81 | +- Prompt templates in `prompts/` directory |
| 82 | +- GitHub Models API for inference |
| 83 | + |
| 84 | +### Dependencies |
| 85 | +- GitHub Models API - Requires `GITHUB_TOKEN` with Models scopes |
| 86 | +- `commander` - CLI argument parsing |
| 87 | +- `dotenv` - Environment variable loading |
| 88 | +- Copilot Spaces (optional) - Can export/import prompts |
| 89 | + |
| 90 | +### Data outputs |
| 91 | +- Refined markdown content (preview or written to files) |
| 92 | +- Diffs showing proposed changes |
| 93 | +- Merged frontmatter updates |
| 94 | + |
| 95 | +## Cross-links & Ownership |
| 96 | + |
| 97 | +### Related subjects |
| 98 | +- [`src/content-render`](../content-render/README.md) - Content parsing and rendering |
| 99 | +- Content files in `content/` - Target of refinements |
| 100 | + |
| 101 | +### Internal documentation |
| 102 | +- [GitHub Models API docs](https://docs.github.com/en/rest/models/inference) |
| 103 | +- Copilot Spaces for prompt management |
| 104 | + |
| 105 | +### Ownership |
| 106 | +- Team: Docs Content (for use and development) |
| 107 | +- Note: These tools are for the docs-content team. We welcome them to use Copilot to support and develop these tools, but docs-engineering is largely hands off. |
| 108 | + |
| 109 | +## Current State & Next Steps |
| 110 | + |
| 111 | +### Available refinement types |
| 112 | + |
| 113 | +Current prompts: |
| 114 | +- `versioning` - Refines version-related content |
| 115 | +- `intro` - Improves article introductions |
| 116 | + |
| 117 | +Each prompt defines: |
| 118 | +- Instructions for the LLM |
| 119 | +- Expected output format |
| 120 | +- Quality criteria |
| 121 | + |
| 122 | +### Adding new refinements |
| 123 | + |
| 124 | +1. Create prompt file in `prompts/` (e.g., `readability.md`). |
| 125 | +2. Write prompt instructions and examples. |
| 126 | +3. Test with Models UI first. |
| 127 | +4. Use `--refine readability` to apply. |
| 128 | + |
| 129 | +Prompt template in `prompts/prompt-template.yml`. |
| 130 | + |
| 131 | +### Copilot Spaces integration |
| 132 | + |
| 133 | +Export Space to prompt: |
| 134 | +```bash |
| 135 | +npm run ai-tools -- --exportSpace space-id --output prompts/my-prompt.md |
| 136 | +``` |
| 137 | + |
| 138 | +Use Space as prompt: |
| 139 | +```bash |
| 140 | +npm run ai-tools -- --space space-id --files content/path/to/file.md |
| 141 | +``` |
| 142 | + |
| 143 | +### Known limitations |
| 144 | +- Requires GitHub token with Models scopes |
| 145 | +- API rate limits apply |
| 146 | +- Quality depends on prompt engineering |
| 147 | +- Currently manual execution (not in CI) |
| 148 | +- No automated testing/evals yet |
| 149 | + |
| 150 | +### Best practices |
| 151 | + |
| 152 | +**Prompt engineering:** |
| 153 | +- Test prompts in GitHub Models UI first |
| 154 | +- Include clear examples in prompts |
| 155 | +- Define expected output format |
| 156 | +- Iterate on prompts based on results |
| 157 | + |
| 158 | +**File selection:** |
| 159 | +- Start with single files to test |
| 160 | +- Use glob patterns for batch processing |
| 161 | +- Preview changes before writing |
| 162 | + |
| 163 | +**Quality assurance:** |
| 164 | +- Always review AI suggestions |
| 165 | +- Don't blindly accept all changes |
| 166 | +- Consider subject matter expertise needed |
| 167 | +- Test refined content for correctness |
| 168 | + |
| 169 | +### Troubleshooting |
45 | 170 |
|
46 | | -* A valid `GITHUB_TOKEN` with Models scopes in your local `.env` file. |
| 171 | +**Missing token error:** |
| 172 | +Add `GITHUB_TOKEN` to `.env` with Models scopes. |
47 | 173 |
|
48 | | -## Future development ideas |
| 174 | +**API errors:** |
| 175 | +- Check token permissions |
| 176 | +- Verify rate limits |
| 177 | +- Check Models API status |
49 | 178 |
|
50 | | -* Add prompts to support all available editors. |
51 | | -* Test prompts in Models UI and add evals to prevent regressions. |
52 | | -* Enable running in CI. |
53 | | -* Explore the new `llm` plugin for GitHub Models (see https://github.com/github/copilot-productivity/discussions/5937). |
54 | | -* Add MCP for more comprehensive context. |
55 | | -* Integrate with Copilot Edit mode in VS Code. |
56 | | -* Add unit tests. |
| 179 | +**Poor refinement quality:** |
| 180 | +- Refine the prompt template |
| 181 | +- Add more examples |
| 182 | +- Test in Models UI first |
| 183 | +- Consider different model/parameters |
0 commit comments