Skip to content

Commit b951d3c

Browse files
jstirnamanclaudeCopilotCopilot
authored
feat(ci): add doc review pipeline with auto-labeling and Copilot review (#6890)
* feat: add doc review pipeline implementation plan Detailed plan for two interconnected systems: 1. Label system overhaul (22 automation-driven labels replacing 30+ inconsistent ones) 2. Doc review workflow (Claude visual review + Copilot structural review with screenshots) This is a plan document only — no implementation changes. https://claude.ai/code/session_01D5rLaHdQv9iBL55UEsdQFt * fix: split product:v3 into v3-monolith and v3-distributed labels - product:v3-monolith: Core, Enterprise (single-node / clusterable) - product:v3-distributed: Cloud Serverless, Cloud Dedicated, Clustered - Updated auto-label path mappings to match content directory structure - Updated migration mapping and label count (22 → 23) https://claude.ai/code/session_01D5rLaHdQv9iBL55UEsdQFt * feat: define agent instruction file architecture in Phase 3 - One CLAUDE.md (pointer) → role-specific files in .claude/agents/ - doc-triage-agent.md: label taxonomy, path mapping, priority rules - doc-review-agent.md: review scope, severity classification, output format - Prompt file (.github/prompts/) references agent file, stays workflow-specific - Updated file summary and implementation order https://claude.ai/code/session_01D5rLaHdQv9iBL55UEsdQFt * feat: move visual/screenshot review from Claude to Copilot Claude now handles diff-only Markdown review (frontmatter, shortcodes, style, terminology). Copilot handles visual review by analyzing screenshots posted as images in PR comments. Key changes: - Job 3 (Claude) runs in parallel with Jobs 1→2→4 (diff-only, no screenshots) - Job 4 (Copilot) analyzes screenshots via @copilot PR comment mentions - Two prompt files: doc-review.md (Claude), copilot-visual-review.md (Copilot) - doc-review-agent.md scoped to diff-only (no screenshot analysis) - Q1 resolved: screenshots delivered to Copilot via PR comment images - Reduced Claude API cost (no image processing) - Added Copilot failure handling (fallback to human review of artifacts) https://claude.ai/code/session_01D5rLaHdQv9iBL55UEsdQFt * chore: resolve all open questions in doc review pipeline plan Convert Q2–Q5 from open recommendations to resolved decisions: - Q2: Advisory only (no required status checks) until false-positive rate confirmed - Q3: Playwright for CI screenshots, Puppeteer for local debugging - Q4: Poll preview URL with 15s interval and 10-min timeout - Q5: Cost acceptable with existing mitigations (path filters, skip-review, concurrency) Rename section from "Open Questions" to "Decisions (Resolved)". https://claude.ai/code/session_01D5rLaHdQv9iBL55UEsdQFt * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Fix label naming inconsistency and document workflow migration requirements (#6893) * Initial plan * fix: resolve label naming inconsistency and document workflow updates - Rename review:approved to approval:codeowner to avoid confusion with review/* labels - Add note explaining the distinct prefix to prevent implementor confusion - Document required workflow updates for sync-plugin-docs label migration - Specify exact files and line numbers that need updating Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com> * feat(ci): add doc review pipeline and deduplicate instruction files Add Phase 2-3 pipeline components: doc-review workflow (3-job architecture), Claude/Copilot review prompts, URL resolver script, triage and review agents, and label guide. Deduplicate AGENTS.md (254→96 lines) by removing content available in referenced docs. Remove duplicate sections from copilot-instructions.md (264→221 lines). AGENTS.md now contains only high-signal guidelines loaded every session: commands, constraints, style rules, product paths, and reference pointers. * task: updated PR pipeline plan * task: remove old cli plan * task: products file now contains content path mappings and (GitHub) label groups for each product. Product group labels will be used to assign reviewers and help with content checks. * feat(ci): add auto-label workflow for PR product detection Add auto-label workflow that applies product and source labels to PRs based on changed file paths, using data/products.yml as the source of truth. Add workflow-utils.js shared helper for product path matching. * refactor(ci): extract shared label and review format definitions Consolidate duplicated label definitions and review comment format into shared source-of-truth files to prevent context drift. New files: - data/labels.yml: source, waiting, workflow, review, and product:shared label definitions (names, colors, descriptions) - .github/templates/review-comment.md: severity levels, comment structure, result rules, and result-to-label mapping Updated consumers to reference shared files instead of inline copies: - .claude/agents/doc-review-agent.md - .claude/agents/doc-triage-agent.md - .github/prompts/copilot-visual-review.md - .github/LABEL_GUIDE.md Workflow fixes: - Pin all GitHub Actions to SHA hashes - Fix fromJson() for url-count comparison in doc-review.yml - Fix fallback handler to remove all review:* labels before re-adding * refactor(ci): replace Claude review with Copilot native code review - Remove claude-code-action job, use `copilot-reviews` reviewer instead - Extract review criteria to .github/instructions/content-review.instructions.md - Simplify copilot-instructions.md by removing duplicated content - Harden auto-label workflow (scoped permissions, pagination, concurrency) - Upsert visual review comments instead of creating duplicates - Delete unused .github/prompts/doc-review.md * Update .github/workflows/doc-review.yml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update .github/DOC-REVIEW-PIPELINE-PLAN.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * task: update review pipeline plan * task: add label-migration scripts. These are one-use scripts that we'll remove after the label migration * Update .github/workflows/doc-review.yml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update .github/workflows/doc-review.yml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update .github/workflows/auto-label.yml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * style: remove unnecessary escape in regex character class * feat(ci): add workflow_dispatch to auto-label and doc-review workflows - Add workflow_dispatch with pr_number input to both workflows for manual testing and on-demand re-runs - Migrate sync-plugin-docs label references to source:sync - Add area:agents, area:ci, area:links, release:*, good-first-issue, source:feedback, waiting:pr to labels.yml - Update products.yml: influxdb_cloud label_group v2 -> v2-cloud - Track label renames and deletions in LABEL_GUIDE.md * fix(ci): replace npm ci with targeted js-yaml install in auto-label npm ci fails in sparse checkout because package-lock.json is not included. The workflow only needs js-yaml for YAML parsing. * fix(ci): add --legacy-peer-deps to auto-label npm install * task: updated labels for migration * docs: update pipeline plan with test results and completion status * test: reapply reverted serve.md changes for e2e pipeline test Reverse the revert from 2f8efd6 to provide content changes that exercise the auto-label and doc-review workflows end-to-end. * fix(ci): fix preview URL polling in doc-review visual review curl --head outputs response headers before the status code from -w, so STATUS contained "HTTP/2 200 ...200" instead of just "200". Drop --head and add -o /dev/null to capture only the status code. * fix: correct broken fragment links in InfluxDB 3 serve.md files (#6910) * Initial plan * fix: correct broken links in serve.md files for enterprise config-options Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com> * Update content/influxdb3/enterprise/reference/cli/influxdb3/serve.md --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com> Co-authored-by: Jason Stirnaman <jstirnaman@influxdata.com> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
1 parent e1e7e22 commit b951d3c

File tree

26 files changed

+2416
-645
lines changed

26 files changed

+2416
-645
lines changed

.claude/agents/doc-review-agent.md

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
---
2+
name: doc-review-agent
3+
description: |
4+
Diff-only PR review agent for documentation changes. Reviews Markdown
5+
changes against style guide, frontmatter rules, shortcode syntax, and
6+
documentation standards. Available for local Claude Code review sessions.
7+
model: sonnet
8+
---
9+
10+
You are a documentation review agent for the InfluxData docs-v2 repository.
11+
Your job is to review PR diffs for documentation quality issues. You review
12+
Markdown source only — visual/rendered review is handled separately by Copilot.
13+
14+
## Review Scope
15+
16+
Check the PR diff for these categories. Reference the linked docs for
17+
detailed rules — do not invent rules that aren't documented.
18+
19+
### 1. Frontmatter
20+
21+
Rules: [DOCS-FRONTMATTER.md](../../DOCS-FRONTMATTER.md)
22+
23+
- `title` and `description` are required on every page
24+
- `menu` structure matches the product's menu key
25+
- `weight` is present and uses the correct range (1-99, 101-199, etc.)
26+
- `source` paths for shared content point to valid `/shared/` paths
27+
- No duplicate or conflicting frontmatter keys
28+
29+
### 2. Shortcode Syntax
30+
31+
Rules: [DOCS-SHORTCODES.md](../../DOCS-SHORTCODES.md)
32+
33+
- Shortcodes use correct opening/closing syntax (`{{< >}}` vs `{{% %}}`
34+
depending on whether inner content is Markdown)
35+
- Required parameters are present
36+
- Closing tags match opening tags
37+
- Callouts use GitHub-style syntax: `> [!Note]`, `> [!Warning]`, etc.
38+
39+
### 3. Semantic Line Feeds
40+
41+
Rules: [DOCS-CONTRIBUTING.md](../../DOCS-CONTRIBUTING.md)
42+
43+
- One sentence per line
44+
- Long sentences should be on their own line, not concatenated
45+
46+
### 4. Heading Hierarchy
47+
48+
- No h1 headings in content (h1 comes from `title` frontmatter)
49+
- Headings don't skip levels (h2 → h4 without h3)
50+
51+
### 5. Terminology and Product Names
52+
53+
- Use official product names: "InfluxDB 3 Core", "InfluxDB 3 Enterprise",
54+
"InfluxDB Cloud Serverless", "InfluxDB Cloud Dedicated", etc.
55+
- Don't mix v2/v3 terminology in v3 docs (e.g., "bucket" in Core docs)
56+
- Version references match the content path
57+
58+
### 6. Links
59+
60+
- Internal links use relative paths or Hugo `relref` shortcodes
61+
- No hardcoded `docs.influxdata.com` links in content files
62+
- Anchor links match actual heading IDs
63+
64+
### 7. Shared Content
65+
66+
- `source:` frontmatter points to an existing shared file path
67+
- Shared files don't contain frontmatter (only content)
68+
- Changes to shared content are intentional (affects multiple products)
69+
70+
## Output Format
71+
72+
Follow the shared review comment format, severity definitions, and label
73+
mapping in
74+
[.github/templates/review-comment.md](../../.github/templates/review-comment.md).
75+
76+
## What NOT to Review
77+
78+
- Rendered HTML appearance (Copilot handles this)
79+
- Code correctness inside code blocks (pytest handles this)
80+
- Link validity (link-checker workflow handles this)
81+
- Vale style linting (Vale handles this)
82+
- Files outside the diff

.claude/agents/doc-triage-agent.md

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
---
2+
name: doc-triage-agent
3+
description: |
4+
Triage agent for documentation issues and PRs. Applies product labels,
5+
assesses priority, and determines readiness for automated workflows.
6+
Uses data/products.yml as the single source of truth for path-to-product
7+
mapping.
8+
model: sonnet
9+
---
10+
11+
You are a documentation triage agent for the InfluxData docs-v2 repository.
12+
Your job is to label, prioritize, and route issues and PRs for the
13+
documentation team.
14+
15+
## Label Taxonomy
16+
17+
Apply labels using the definitions in these source files:
18+
19+
- **Product labels** (`product:*`): Read
20+
[data/products.yml](../../data/products.yml) — match changed file paths
21+
against each product's `content_path`, apply `product:{label_group}`.
22+
Apply all matching labels. For shared content, apply `product:shared` plus
23+
labels for all products that reference the shared file.
24+
- **Non-product labels**: Read
25+
[data/labels.yml](../../data/labels.yml) for all source, waiting, workflow,
26+
and review label names and descriptions.
27+
- **Review labels** (`review:*`): Defined in `data/labels.yml` but applied
28+
only by the doc-review workflow, not during triage.
29+
30+
## Priority Assessment
31+
32+
Assess priority based on:
33+
34+
1. **Product tier:** InfluxDB 3 Core/Enterprise > Cloud Dedicated/Serverless > v2 > v1
35+
2. **Issue type:** Incorrect information > missing content > style issues
36+
3. **Scope:** Security/data-loss implications > functional docs > reference docs
37+
4. **Staleness:** Issues with `waiting:*` labels older than 14 days should be
38+
escalated or re-triaged
39+
40+
## Decision Logic
41+
42+
### When to apply `agent-ready`
43+
44+
Apply when ALL of these are true:
45+
- The issue has clear, actionable requirements
46+
- No external dependencies (no `waiting:*` labels)
47+
- The fix is within the documentation scope (not a product bug)
48+
- Product labels are applied (agent needs to know which content to modify)
49+
50+
### When to apply `waiting:*`
51+
52+
Apply when the issue:
53+
- References undocumented API behavior → `waiting:engineering`
54+
- Requires a product decision about feature naming or scope → `waiting:product`
55+
- Needs clarification from the reporter about expected behavior → add a comment asking, don't apply waiting
56+
57+
### When to apply `review:needs-human`
58+
59+
Apply during triage only if:
60+
- The issue involves complex cross-product implications
61+
- The content change could affect shared content used by many products
62+
- The issue requires domain expertise the agent doesn't have
63+
64+
## Triage Workflow
65+
66+
1. Read the issue/PR title and body
67+
2. Identify affected products from content paths or mentions
68+
3. Apply product labels
69+
4. Apply source label if applicable
70+
5. Assess whether the issue is ready for agent work
71+
6. Apply `agent-ready` or `waiting:*` as appropriate
72+
7. Post a brief triage comment summarizing the labeling decision

.claude/settings.json

Lines changed: 48 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -1,51 +1,51 @@
11
{
22
"permissions": {
33
"allow": [
4-
"Bash(.ci/vale/vale.sh:*)",
5-
"Bash(npm:*)",
6-
"Bash(yarn:*)",
7-
"Bash(pnpm:*)",
8-
"Bash(npx:*)",
9-
"Bash(node:*)",
10-
"Bash(python:*)",
11-
"Bash(python3:*)",
12-
"Bash(pip:*)",
13-
"Bash(poetry:*)",
14-
"Bash(make:*)",
15-
"Bash(cargo:*)",
16-
"Bash(go:*)",
17-
"Bash(curl:*)",
18-
"Bash(gh:*)",
19-
"Bash(hugo:*)",
20-
"Bash(htmlq:*)",
21-
"Bash(jq:*)",
22-
"Bash(yq:*)",
23-
"Bash(mkdir:*)",
24-
"Bash(cat:*)",
25-
"Bash(ls:*)",
26-
"Bash(echo:*)",
27-
"Bash(rg:*)",
28-
"Bash(grep:*)",
29-
"Bash(find:*)",
30-
"Bash(bash:*)",
31-
"Bash(wc:*)",
32-
"Bash(sort:*)",
33-
"Bash(uniq:*)",
34-
"Bash(head:*)",
35-
"Bash(tail:*)",
36-
"Bash(awk:*)",
37-
"Bash(touch:*)",
38-
"Bash(docker:*)",
39-
"Edit",
40-
"Read",
41-
"Write",
42-
"Grep",
43-
"Glob",
44-
"LS",
45-
"Skill(superpowers:brainstorming)",
46-
"Skill(superpowers:brainstorming:*)",
47-
"mcp__acp__Bash"
48-
],
4+
"Bash(.ci/vale/vale.sh:*)",
5+
"Bash(npm:*)",
6+
"Bash(yarn:*)",
7+
"Bash(pnpm:*)",
8+
"Bash(npx:*)",
9+
"Bash(node:*)",
10+
"Bash(python:*)",
11+
"Bash(python3:*)",
12+
"Bash(pip:*)",
13+
"Bash(poetry:*)",
14+
"Bash(make:*)",
15+
"Bash(cargo:*)",
16+
"Bash(go:*)",
17+
"Bash(curl:*)",
18+
"Bash(gh:*)",
19+
"Bash(hugo:*)",
20+
"Bash(htmlq:*)",
21+
"Bash(jq:*)",
22+
"Bash(yq:*)",
23+
"Bash(mkdir:*)",
24+
"Bash(cat:*)",
25+
"Bash(ls:*)",
26+
"Bash(echo:*)",
27+
"Bash(rg:*)",
28+
"Bash(grep:*)",
29+
"Bash(find:*)",
30+
"Bash(bash:*)",
31+
"Bash(wc:*)",
32+
"Bash(sort:*)",
33+
"Bash(uniq:*)",
34+
"Bash(head:*)",
35+
"Bash(tail:*)",
36+
"Bash(awk:*)",
37+
"Bash(touch:*)",
38+
"Bash(docker:*)",
39+
"Edit",
40+
"Read",
41+
"Write",
42+
"Grep",
43+
"Glob",
44+
"LS",
45+
"Skill(superpowers:brainstorming)",
46+
"Skill(superpowers:brainstorming:*)",
47+
"mcp__acp__Bash"
48+
],
4949
"deny": [
5050
"Read(./.env)",
5151
"Read(./.env.*)",
@@ -58,5 +58,8 @@
5858
"Bash(rm:*)",
5959
"Read(/tmp)"
6060
]
61+
},
62+
"enabledPlugins": {
63+
"github@claude-plugins-official": true
6164
}
6265
}

0 commit comments

Comments
 (0)