feat: implement artifact-based documentation aggregation system with repository_dispatch

[Copilot]

• Analyze current repository structure and documentation generation system
• Test current build and dependencies
• Design documentation aggregation system (pull mode)
• Create script to fetch documentation from project repositories
• Add configuration file for documentation sources
• Update generate-docs.js script path in package.json
• Add support for documentation metadata (version, source repo)
• Create documentation structure for aggregated content
• Update GitHub Actions workflow to use new system
• Add documentation about the system
• Fix NODE_PATH for script execution
• Fix broken links in documentation
• Verify build works with new changes
• Fix security issue with incomplete sanitization
• Add usage examples
• Run security checks (CodeQL + dependency checks)
• Implement push mode with reusable workflow
• Add example workflow template for projects
• Update documentation for both modes
• Remove pull mode per feedback - keep only push mode
• Refactor to repository_dispatch architecture
• Switch to artifact-based transfer
• Address PR feedback (convert to github-script, use ci-common)
• Simplify workflow inputs
• Final refactoring per feedback

Changes in This Commit

Final round of improvements based on PR feedback:

1. Converted Dispatcher to GitHub Script

• Prepare documentation bundle step now uses actions/github-script@v7
• Replaced bash script with JavaScript for better maintainability
• Handles frontmatter injection, file copying, and metadata in pure Node.js
• More reliable and easier to debug

2. Simplified Receiver Workflow

• Removed build step - build/validation handled by push to main workflow
• Removed change detection - handled by create-and-merge-pull-request action
• Removed deployment step - deployment handled by update-docs.yml workflow
• Removed unused branch variable - not needed after simplification
• Workflow now focuses solely on injecting documentation and creating PR

3. Clean Separation of Concerns

• Receiver workflow: Download artifact → Inject docs → Create PR
• Update-docs workflow: Build documentation → Deploy to Pages
• Each workflow has a single, clear responsibility
• No duplication of build/deploy logic

4. Updated Documentation

• README reflects new simplified architecture
• Build and deployment now clearly attributed to push to main workflow
• Updated troubleshooting section with correct workflow references

Benefits

• ✅ Cleaner workflow logic (single responsibility)
• ✅ No duplicate build steps
• ✅ Consistent use of github-script throughout
• ✅ Easier to maintain and debug
• ✅ Better separation of concerns
• ✅ Leverages existing push to main workflow for build/deploy

Original prompt

As this repo must centralize the documentation of every Hoverkraft projects we must find a way to get (pull or push mode) the documentation coming from every project.

It is important that documentation source for each project stay in the same repository as the project to keep them atomic. But we need to merge it in a single point at the end.

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[neilime]

@copilot IMO push mode is better because documentation ca be updated in real time.

Maybe public-docs can provide an reusable workflow that can be used by the repository we want to sync with public-docs

[Copilot]

@copilot IMO push mode is better because documentation ca be updated in real time.

Maybe public-docs can provide an reusable workflow that can be used by the repository we want to sync with public-do...

Agreed! Push mode has been implemented via a reusable workflow that projects can use for real-time documentation updates.

Implementation (commit 7cdd735):

Added .github/workflows/sync-docs-push.yml - a reusable workflow that project repositories can call to push their documentation immediately when changes occur.

Usage in project repositories:

name: Push Documentation to Portal

on:
  push:
    branches: [main]
    paths: ['docs/**', 'README.md']

jobs:
  push-docs:
    uses: hoverkraft-tech/public-docs/.github/workflows/sync-docs-push.yml@main
    with:
      source_repo: 'your-repo-name'
      target_path: 'projects/your-repo-name'
    secrets:
      PUBLIC_DOCS_TOKEN: ${{ secrets.PUBLIC_DOCS_TOKEN }}

Key features:

• Real-time sync on every commit to documentation
• Automatic frontmatter injection with source metadata
• No waiting for scheduled pulls
• Works alongside pull mode for redundancy

See the template at .github/workflows/EXAMPLE-push-docs.yml.template and updated documentation in application/docs/documentation-system.md.

[neilime]

@copilot you can remove pull mode

[Copilot]

@copilot you can remove pull mode

Done! Pull mode has been removed (commit 675b036).

The system now uses push mode only with these changes:

Removed:

• .github/scripts/pull-docs.js
• .github/docs-sources.yml
• pull-docs and sync-docs npm scripts
• js-yaml dependency
• All pull mode documentation

Updated:

• Workflows simplified to metadata generation only
• Documentation focused entirely on push mode
• Examples show only push workflow usage

Projects now simply add a workflow to push their documentation in real-time when changes occur.

[neilime]

@copilot this repo shoudl have two workflow for synd docs:

• One that is a repository dispatch triiger that can be call by other repo to update their related documentaiotn section.
• One that is a resuable workflow to used byu repository to helps them calling the first repository_dispatch workflow.

The repository should pass the documentaiton to update, the public-ocs shoudl inject it in its documentaiton, then build an d publish if all is ok

[Copilot]

@copilot this repo shoudl have two workflow for synd docs:

• One that is a repository dispatch triiger that can be call by other repo to update their related documentaiotn section.
• One that is a res...

Done! Implemented repository_dispatch architecture (commit a7d36a8).

New architecture:

Workflow 1: Dispatcher (sync-docs-dispatcher.yml - reusable)

• Called by project repositories
• Bundles documentation with source metadata
• Sends via repository_dispatch event to public-docs

Workflow 2: Receiver (sync-docs-receiver.yml - in public-docs)

• Listens for repository_dispatch type sync-documentation
• Receives and injects documentation
• Builds to validate before committing
• Only publishes if build succeeds

Benefits:

• Build validation prevents broken documentation
• Projects only need repo scope token (for repository_dispatch)
• Failed builds don't corrupt the site
• Documentation passed as payload, public-docs controls injection and publishing

Projects use it like:

jobs:
  push-docs:
    uses: hoverkraft-tech/public-docs/.github/workflows/sync-docs-dispatcher.yml@main
    with:
      source_repo: 'my-project'
      target_path: 'projects/my-project'
    secrets:
      PUBLIC_DOCS_TOKEN: ${{ secrets.PUBLIC_DOCS_TOKEN }}

[neilime]

@copilot is possible to pass an artifact from the doc sources to the public-docs?

[Copilot]

@copilot is possible to pass an artifact from the doc sources to the public-docs?

Yes, it's possible! Using artifacts would be an alternative approach to the current base64-encoded payload in repository_dispatch.

Current approach: Documentation is bundled, compressed, and base64-encoded in the dispatcher workflow, then passed via repository_dispatch payload (limited to ~64KB).

Artifact approach would:

1. Dispatcher: Upload documentation as artifact
2. Dispatcher: Send repository_dispatch with artifact name/run_id
3. Receiver: Download artifact using the artifact name/run_id
4. Receiver: Extract and inject documentation

Trade-offs:

Current (payload):

• ✅ Simpler - everything in one event
• ✅ Faster - no artifact upload/download
• ❌ Size limit (~64KB for payloads)

Artifacts:

• ✅ No size limit (artifacts support large files)
• ✅ Better for image-heavy documentation
• ❌ More complex - requires artifact download step
• ❌ Cross-repo artifact access needs proper token permissions
• ❌ Artifacts expire after 90 days by default

Would you like me to implement the artifact-based approach for handling larger documentation bundles?

[Copilot]

@copilot uses artifacts

Done! Switched to artifact-based transfer (commit c606ab8).

Changes:

Dispatcher workflow:

• Uploads documentation as GitHub Actions artifact (retention: 1 day)
• Sends artifact metadata via repository_dispatch (name, run_id, repository)
• No more base64 encoding

Receiver workflow:

• Downloads artifact from source repository
• Extracts documentation directly
• No more base64 decoding

Benefits:

• ✅ No size limits (artifacts support up to 10GB vs ~64KB payload limit)
• ✅ Better for image-heavy documentation
• ✅ Cleaner implementation
• ✅ More reliable

Same token permissions needed (repo scope for both repository_dispatch and artifact access).

[neilime]

@copilot handle feedbacks

[Copilot]

@copilot handle feedbacks

All feedback addressed in commit 98d2dda:

• ✅ Moved documentation-system.md to README.md
• ✅ Converted bash scripts to github-script steps
• ✅ Using hoverkraft-tech/ci-github-common for PR creation
• ✅ Using core.summary for cleaner output

Build tested successfully.

[neilime]

@copilot Handle feedbacks

[Copilot]

@copilot Handle feedbacks

All feedback addressed in commit 538854f:

• ✅ Removed target_path input (auto-calculated as projects/{source_repo})
• ✅ Removed branch input (always syncs default branch)
• ✅ Converted summary to github-script with core.summary
• ✅ Updated documentation and examples

Build tested successfully.

[neilime]

@copilot Handle feedbacks

[Copilot]

@copilot Handle feedbacks

All feedback addressed in commit 8c2e0d5:

• ✅ Dispatcher: Converted bash script to actions/github-script@v7
• ✅ Receiver: Removed unused branch variable
• ✅ Receiver: Removed build and change detection steps (handled by push to main workflow)
• ✅ Receiver: Removed deployment step (handled by update-docs.yml workflow)
• ✅ Updated documentation to reflect new simplified architecture

The receiver workflow now focuses solely on downloading artifacts and creating PRs. Build and deployment are handled by the existing update-docs.yml workflow when changes are pushed to main.