Skip to content

auto-triage bot improvements#15513

Open
FredKSchott wants to merge 12 commits intomainfrom
fks/triage-6
Open

auto-triage bot improvements#15513
FredKSchott wants to merge 12 commits intomainfrom
fks/triage-6

Conversation

@FredKSchott
Copy link
Member

@FredKSchott FredKSchott commented Feb 14, 2026

Lots of small improvements as we finally start to get some broken / incomplete triage runs back from the auto-triage bot.

Summary

  • New: Add a verify step to the triage pipeline that checks whether reported behavior is intentional before attempting a fix. Fixes issues where the bot just trusted the submitting user's expected behavior as truth vs. potentially confused/incorrect on expected behavior.
  • New: Make diagnose and fix skills aware of the withastro/compiler repo (cloned as a sibling in CI). Fixes issues tracked back to the compiler, where the bot was trying to work around the issue in our astro codebase instead of pointing responsibility to the compiler.
  • New: Add a feasibility check to the fix skill for browser/runtime compatibility. Hopefully fixes issues where the bot suggests code that wouldn't run on modern browsers.
  • Fix: For some reason the reproduction instructions were gone (or never there?) so we hadn't been downloading repos/stackblitz, and probably spending quite a lot of time trying to figure out the bug without a reproduction. Kind of surprised by the success rate at reproductions, given this, but I guess everyone is including enough detail without it for the LLM to go off of.
  • Chore: Ensure all skills explicitly read report.md before appending to it
  • Chore: Simplify the diagnose skill's review step
  • Chore: Refactor issue-triage.ts into composable helper functions.
  • Chore: Ignore triage folder from eslint
  • Chore: Tidy up AGENTS.md, simplified the project layout section
  • Chore: Bump @flue/cli to 0.0.20 and @flue/client to 0.0.12

Testing

No good way to test CI locally, so will need to test a bit post-merge.

Fred K. Schott added 10 commits February 13, 2026 21:12
Adds a new verification phase between diagnose and fix that researches
whether reported behavior is an actual bug or intended design. This
prevents wasting effort attempting fixes for non-bugs.
The withastro/compiler repo may be cloned as a sibling directory.
Instructs the diagnose skill to check it when stack traces point to
compiler behavior, and the fix skill to document proposed compiler
changes in report.md.
Adds a new step to verify browser/runtime compatibility before
implementing a fix. Also adds compiler repo awareness and renumbers
steps accordingly.
Adds verifyResultSchema and runs verify after diagnose. Skips the fix
step when verification determines behavior is intended. Also clones the
compiler repo in CI so diagnose/fix skills can reference it.
Adds explicit 'read report.md' to the critical instruction at the top
of every skill file, so agents always load prior context before
appending their own findings.
The bullet list was redundant — the agent reads report.md and extracts
whatever context it needs. Replace with a single instruction.
Extracts shouldRetriage, selectTriageLabels, fetchIssue, and
runTriagePipeline from the monolithic triage function. Inlines schemas
next to their call sites, adds early returns for non-reproducible and
intended-behavior cases, and validates the issue response with valibot.
@changeset-bot
Copy link

changeset-bot bot commented Feb 14, 2026

⚠️ No Changeset found

Latest commit: 4f5a134

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@FredKSchott FredKSchott changed the title Fks/triage 6 auto-triage bot improvements Feb 14, 2026
@github-actions github-actions bot added the 🚨 action Modifies GitHub Actions label Feb 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🚨 action Modifies GitHub Actions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant