Future Consideration: AI-Powered Summaries via Copilot CLI Integration

[mheadd]

Summary

Explore an alternative/complementary approach that uses gh copilot CLI to generate AI-powered summaries of specification changes, rather than (or in addition to) showing raw diffs.

Current Approach

The current action is a self-contained TypeScript GitHub Action that:

• Detects spec file changes via git
• Generates diffs programmatically
• Renders issues using Handlebars templates
• Creates GitHub issues via the API

Alternative Approach: Git Diff + Copilot CLI

git diff HEAD~1 -- docs/specification.md | gh copilot explain

Pros

• AI-powered summaries: Can explain changes in natural language rather than showing raw diffs
• Flexible: Natural language prompts are highly adaptable
• Quick prototyping: Fast to set up for single-repo use cases

Cons

• Not Marketplace-distributable: Shell scripts are not packageable as reusable actions
• Authentication complexity: Requires gh CLI with Copilot extension and user-level auth (no GITHUB_TOKEN support)
• Runner setup: gh copilot not installed by default; requires gh extension install github/gh-copilot
• Non-deterministic: LLM output varies; hard to test or guarantee consistent formatting
• Cost: Incurs Copilot API usage costs at scale
• Enterprise restrictions: Many orgs block or restrict Copilot usage
• Security: Prompt injection risk from adversarial diff content

Recommendation

The current self-contained approach remains best for a shareable, Marketplace-ready action. However, AI summaries could be added as an opt-in enhancement in a future version:

- uses: spec-ops-method/spec-ops-action@v2
  with:
    ai-summary: true  # Optional: Use AI to summarize changes
    ai-provider: copilot  # or: openai, anthropic

Implementation Considerations

If pursued, an AI summary feature would need to:

1. Be optional with a fallback to raw diffs
2. Handle authentication gracefully (fail soft if not configured)
3. Sanitize inputs to prevent prompt injection
4. Document costs clearly to users
5. Support multiple providers for flexibility