Skip to content

feat: nightly AI scanner workflows (code review, docs, autonomous scanner)#692

Merged
Lightheartdevs merged 3 commits intoLight-Heart-Labs:mainfrom
Tony363:feat/ai-nightly-scanners
Apr 2, 2026
Merged

feat: nightly AI scanner workflows (code review, docs, autonomous scanner)#692
Lightheartdevs merged 3 commits intoLight-Heart-Labs:mainfrom
Tony363:feat/ai-nightly-scanners

Conversation

@Tony363
Copy link
Copy Markdown
Contributor

@Tony363 Tony363 commented Mar 31, 2026

Summary

Adds 3 scheduled AI-powered workflows (split from #683, PR B per review feedback):

  • nightly-code-review: Daily code improvements via Claude ($3-8/run, 3 AM UTC)
  • nightly-docs-update: Daily documentation sync ($1-3/run, 4 AM UTC)
  • autonomous-code-scanner: Multi-pass scanner with $100/run budget cap (2 AM UTC)

Supporting files

  • .github/scripts/anthropic_helper.py — Anthropic API wrapper
  • .github/scripts/generate-type-hints.py + apply-type-hints.py
  • .github/scripts/generate-docstrings.py + apply-docstrings.py
  • .github/prompts/nightly-code-review.md + nightly-docs-update.md

Security hardening

  • All actions pinned to commit SHA
  • GITHUB_TOKEN used for all PR creation (no PAT_TOKEN)
  • Secret validation uses env-block pattern
  • Protected file patterns block installers/, dream-cli, config/

Note: AI-generated draft PRs (from scanners, nightly reviews) use GITHUB_TOKEN
for PR creation. By design, these PRs won't have CI checks run automatically.
A maintainer must interact (push, close/reopen, or label) to trigger CI.

Setup

Only one secret needed: ANTHROPIC_API_KEY

All workflows support workflow_dispatch with dry_run option for testing.

Cost estimate

Monthly: $270–$2,130 depending on scanner depth. Start with P0+P1 only.

Depends on

Test plan

  • CI checks pass
  • actionlint validates all workflow YAML files
  • No mutable action tags or PAT_TOKEN references
  • Manual workflow_dispatch with dry_run: true succeeds

🤖 Generated with Claude Code

Copy link
Copy Markdown
Collaborator

@Lightheartdevs Lightheartdevs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Audit Review

Good architecture — no user-controlled input in prompts, three-layer protected file enforcement, budget caps, SHA-pinned actions, GITHUB_TOKEN for all PR creation. However, three blocking issues:

Blocking: npx -y @anthropic-ai/claude-code is not version-pinned

Both nightly-code-review.yml and nightly-docs-update.yml install Claude Code via npx -y @anthropic-ai/claude-code with no version specifier. This always fetches latest from npm. A supply chain compromise of the npm package gets arbitrary code execution with ANTHROPIC_API_KEY and GITHUB_TOKEN.

Fix: Pin to a specific version, e.g., npx -y @anthropic-ai/claude-code@0.2.72

Blocking: Cost risk — disable cron triggers initially

Estimated $270-2,130/month on daily schedules. The autonomous scanner alone can hit $100/run — at daily that's potentially $3K/month.

Fix: Change schedule triggers to workflow_dispatch only. Run manually for a few weeks to observe actual costs before enabling cron.

Blocking: Patch collision in create-prs job

The create-prs job applies formatting, type-hints, and documentation patches sequentially on the same checkout, then creates separate PRs per category. But peter-evans/create-pull-request branches from the current accumulated state, so:

  • Formatting PR: clean (only formatting changes)
  • Type-hints PR: contains formatting + type-hints
  • Docs PR: contains formatting + type-hints + docs

Fix: Reset the checkout between each category (git checkout -- . or re-checkout) before applying each patch.

Medium: Deprecated ::set-output

generate-docstrings.py and generate-type-hints.py use ::set-output which is deprecated. Use >> $GITHUB_OUTPUT instead.

Required: Remove gpu.py change

Same fix as #691 and #693. PR #715 already merged this. Remove to avoid conflicts.

Good

  • No user-controlled input in any prompt (prompts from .github/prompts/, inputs are git log and AST)
  • Three-layer protected file enforcement (prompt + post-hoc revert + secret scan)
  • Budget cap ($100/run for autonomous scanner)
  • All actions SHA-pinned
  • GITHUB_TOKEN for PR creation (no recursion)
  • Schedule-only triggers — forks can't trigger
  • Deduplication checks for existing open PRs

Tony363 added a commit to Tony363/DreamServer that referenced this pull request Apr 1, 2026
- Pin @anthropic-ai/claude-code to v2.1.89 (supply chain hardening)
- Disable cron triggers, workflow_dispatch only until cost validated
- Reset checkout between patch categories to prevent collision
- Replace deprecated ::set-output with $GITHUB_OUTPUT in Python scripts
- Remove gpu.py cherry-pick (merged via PR Light-Heart-Labs#715)

Addresses review: PR Light-Heart-Labs#692 pullrequestreview-4047304987

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Tony363 Tony363 force-pushed the feat/ai-nightly-scanners branch from 9c097db to ab4d0b2 Compare April 1, 2026 23:46
Tony363 and others added 2 commits April 1, 2026 20:10
Add 3 scheduled AI-powered workflows:
- nightly-code-review: daily code improvements via Claude ($3-8/run)
- nightly-docs-update: daily documentation sync ($1-3/run)
- autonomous-code-scanner: multi-pass scanner with $100/run budget cap
  (formatting, security, type hints, docstrings)

Supporting files:
- .github/scripts/: anthropic_helper.py + 4 scanner helper scripts
- .github/prompts/: nightly review and docs update prompts

Security hardening:
- All actions pinned to commit SHA
- GITHUB_TOKEN used for all PR creation (no PAT_TOKEN)
- Secret validation uses env-block pattern
- Protected file patterns prevent changes to installers, CLI, config

These workflows run on cron schedule + manual workflow_dispatch.
Estimated cost: $270-2,130/month depending on scanner depth.
Ship when budget allows.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Pin @anthropic-ai/claude-code to v2.1.89 (supply chain hardening)
- Disable cron triggers, workflow_dispatch only until cost validated
- Reset checkout between patch categories to prevent collision
- Replace deprecated ::set-output with $GITHUB_OUTPUT in Python scripts
- Remove gpu.py cherry-pick (merged via PR Light-Heart-Labs#715)

Addresses review: PR Light-Heart-Labs#692 pullrequestreview-4047304987

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Tony363 Tony363 force-pushed the feat/ai-nightly-scanners branch from ab4d0b2 to 5f0a762 Compare April 2, 2026 00:10
…ation

- Move all ${{ }} expressions from shell run blocks into env vars (cost-tracker,
  budget-gates, verify-changes, label-PR steps, final-summary)
- Add default values for potentially empty cost outputs (${VAR:-0})
- Quote PR numbers in gh pr edit commands

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Lightheartdevs Lightheartdevs merged commit 457f713 into Light-Heart-Labs:main Apr 2, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants