Skip to content

auto-triage bot improvements#15513

Merged
FredKSchott merged 40 commits intomainfrom
fks/triage-6
Feb 15, 2026
Merged

auto-triage bot improvements#15513
FredKSchott merged 40 commits intomainfrom
fks/triage-6

Conversation

@FredKSchott
Copy link
Member

@FredKSchott FredKSchott commented Feb 14, 2026

Lots of small improvements as we finally start to get some broken / incomplete triage runs back from the auto-triage bot.

Summary

  • New: Docker sandbox for issue triage: Run the LLM (OpenCode server) inside an isolated Docker container during triage workflows so untrusted reproduction code never has access to secrets. Adds a Dockerfile.sandbox, a GHCR build workflow, and updates the triage workflow to use --sandbox. Moves the compiler clone into .compiler/ (gitignored) so it's accessible inside the container's bind mount.
  • New: Add a verify step to the triage pipeline that checks whether reported behavior is intentional before attempting a fix. Fixes issues where the bot just trusted the submitting user's expected behavior as truth vs. potentially confused/incorrect on expected behavior.
  • New: Make diagnose and fix skills aware of the withastro/compiler repo (cloned as a sibling in CI). Fixes issues tracked back to the compiler, where the bot was trying to work around the issue in our astro codebase instead of pointing responsibility to the compiler.
  • New: Add a feasibility check to the fix skill for browser/runtime compatibility. Hopefully fixes issues where the bot suggests code that wouldn't run on modern browsers.
  • Fix: For some reason the reproduction instructions were gone (or never there?) so we hadn't been downloading repos/stackblitz, and probably spending quite a lot of time trying to figure out the bug without a reproduction. Kind of surprised by the success rate at reproductions, given this, but I guess everyone is including enough detail without it for the LLM to go off of.
  • Chore: Ensure all skills explicitly read report.md before appending to it
  • Chore: Simplify the diagnose skill's review step
  • Chore: Refactor issue-triage.ts into composable helper functions.
  • Chore: Ignore triage folder from eslint
  • Chore: Tidy up AGENTS.md, simplified the project layout section
  • Chore: Bump @flue/cli to 0.0.20 and @flue/client to 0.0.12

Testing

No good way to test CI locally, so will need to test a bit post-merge.

Fred K. Schott added 10 commits February 13, 2026 21:12
Adds a new verification phase between diagnose and fix that researches
whether reported behavior is an actual bug or intended design. This
prevents wasting effort attempting fixes for non-bugs.
The withastro/compiler repo may be cloned as a sibling directory.
Instructs the diagnose skill to check it when stack traces point to
compiler behavior, and the fix skill to document proposed compiler
changes in report.md.
Adds a new step to verify browser/runtime compatibility before
implementing a fix. Also adds compiler repo awareness and renumbers
steps accordingly.
Adds verifyResultSchema and runs verify after diagnose. Skips the fix
step when verification determines behavior is intended. Also clones the
compiler repo in CI so diagnose/fix skills can reference it.
Adds explicit 'read report.md' to the critical instruction at the top
of every skill file, so agents always load prior context before
appending their own findings.
The bullet list was redundant — the agent reads report.md and extracts
whatever context it needs. Replace with a single instruction.
Extracts shouldRetriage, selectTriageLabels, fetchIssue, and
runTriagePipeline from the monolithic triage function. Inlines schemas
next to their call sites, adds early returns for non-reproducible and
intended-behavior cases, and validates the issue response with valibot.
@changeset-bot
Copy link

changeset-bot bot commented Feb 14, 2026

⚠️ No Changeset found

Latest commit: d5bcddb

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@FredKSchott FredKSchott changed the title Fks/triage 6 auto-triage bot improvements Feb 14, 2026
@github-actions github-actions bot added the 🚨 action Modifies GitHub Actions label Feb 14, 2026
AhmadYasser1 and others added 2 commits February 14, 2026 16:44
* fix(markdoc): sync custom attributes between tags and nodes with shared names

In Markdoc, `table` exists as both a tag (`{% table %}`) and a node
(the inner table structure). When users configure custom attributes on
`nodes.table` or `tags.table`, the AST propagates those attributes to
both the tag and node, but validation only checks the schema for each
type independently. This caused "Invalid attribute" errors when
attributes were declared on only one side.

Add `syncTagNodeAttributes()` to automatically merge attribute
declarations between tags and nodes that share the same name after
config setup, so users can define attributes on either side.

Fixes #14220

* chore: clarify why explicit types are needed on builtinTags/builtinNodes
@github-actions github-actions bot added the pkg: integration Related to any renderer integration (scope) label Feb 15, 2026
Fred K. Schott and others added 11 commits February 14, 2026 16:58
…m sandbox

The sandbox subagents had no access to issue data because the orchestrator
fetched it but never passed it through. This caused the reproduce skill to
attempt gh CLI calls which fail without a token in the sandbox.

- Extract IssueDetails valibot schema and type in issue-triage.ts
- Fetch additional fields (author, labels, state, authorAssociation, etc.)
- Pass issueDetails as args to reproduce, diagnose, verify, and fix skills
- Add issueDetails prerequisite to diagnose, verify, and fix skill docs
- Replace gh CLI commands in sandbox-run skills with curl/git alternatives
- Use author_association field for maintainer detection instead of gh api
- Remove gh CLI from sandbox Dockerfile (not usable without token)
…to comment skill

The comment skill now receives available priority labels and selects
one as part of rendering the comment. This makes the priority judgment
visible in the posted comment (answering 'how bad is it?') and lets
the downstream label selector simply extract it rather than deciding
independently.

- Extract fetchRepoLabels helper with valibot validation
- Pass priorityLabels to comment skill, packageLabels to label selector
- Simplify selectTriageLabels to extract priority from comment + pick packages
- Add assert helper for runtime invariant checks
- Validate flue.args with valibot instead of unsafe type cast
- Add v.nonEmpty() to label result schema, remove dead null guard
- Assert fetched label arrays are non-empty before proceeding
@github-actions github-actions bot removed the pkg: integration Related to any renderer integration (scope) label Feb 15, 2026
@FredKSchott FredKSchott merged commit 40f10bb into main Feb 15, 2026
21 checks passed
@FredKSchott FredKSchott deleted the fks/triage-6 branch February 15, 2026 05:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🚨 action Modifies GitHub Actions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants