Add build-duty skill for PR triage across dotnet repos by marcpopMSFT · Pull Request #53311 · dotnet/sdk

marcpopMSFT · 2026-03-06T18:30:19Z

New version of #52678 using the new skills that were introduced.

Creates a new skill under .claude/skills/build-duty/ that helps build duty engineers triage automated PRs across dotnet/sdk, dotnet/installer, dotnet/templating, and dotnet/dotnet repositories.

The skill includes:

Get-BuildDutyReport.ps1: PowerShell script that queries GitHub via gh CLI for PRs from monitored authors (dotnet-maestro[bot], github-actions[bot] merge PRs, vseanreesermsft, dotnet-bot), classifies them into categories (Ready to Merge, Branch Lockdown, Changes Requested, Failing/Blocked), and outputs both human-readable tables and structured JSON.
SKILL.md: Skill definition with workflow instructions for running the script, interpreting results, investigating failures via the ci-analysis skill, and generating a formatted triage report.

Key design decisions:

Uses gh CLI for deterministic, testable queries (no LLM drift)
Uses GraphQL for efficient batching of mergeStateStatus and statusCheckRollup in a single call per PR
Delegates CI failure analysis to the existing ci-analysis skill
Script handles filtering/classification; agent handles presentation

Supersedes the draft approach in PR #52678 which used a pure prompt-based approach that was unreliable.

Creates a new skill under .claude/skills/build-duty/ that helps build duty engineers triage automated PRs across dotnet/sdk, dotnet/installer, dotnet/templating, and dotnet/dotnet repositories. The skill includes: - Get-BuildDutyReport.ps1: PowerShell script that queries GitHub via gh CLI for PRs from monitored authors (dotnet-maestro[bot], github-actions[bot] merge PRs, vseanreesermsft, dotnet-bot), classifies them into categories (Ready to Merge, Branch Lockdown, Changes Requested, Failing/Blocked), and outputs both human-readable tables and structured JSON. - SKILL.md: Skill definition with workflow instructions for running the script, interpreting results, investigating failures via the ci-analysis skill, and generating a formatted triage report. Key design decisions: - Uses gh CLI for deterministic, testable queries (no LLM drift) - Uses GraphQL for efficient batching of mergeStateStatus and statusCheckRollup in a single call per PR - Delegates CI failure analysis to the existing ci-analysis skill - Script handles filtering/classification; agent handles presentation Supersedes the draft approach in PR #52678 which used a pure prompt-based approach that was unreliable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

marcpopMSFT · 2026-03-06T18:34:04Z

Example report. It is a good first step but will need to utilize the helix/CI skills better as some of the next step details are not great.

🔧 Build Duty Triage Report

Date: March 6, 2026
Repositories: dotnet/sdk, dotnet/installer, dotnet/templating, dotnet/dotnet

✅ Ready to Merge (0)

No PRs are currently ready to merge.

🔒 Branch Lockdown (0)

No branches are currently locked down.

⚠️ Changes Requested (0)

No PRs have pending change requests.

❌ Failing / Blocked (17)

dotnet/sdk — Codeflow PRs (4)

#	Title	Target	Age	Checks	Issue
#53250	Source code updates from dotnet/dotnet	main	2d	FAILURE	StaticWebAssets tests failing cross-platform (Windows, macOS, Linux). Dependency update likely broke SWA tests.
#53284	Source code updates from dotnet/dotnet	release/10.0.2xx	<1d	FAILURE	425+ test failures across NetAnalyzers, EndToEnd, Watch, dotnet.Tests. Broad dependency/version issue. Known issue #40006 (watch tests) partially applies.
#53267	Source code updates from dotnet/dotnet	release/11.0.1xx-preview2	1d	FAILURE	7/17 jobs failing — StaticWebAssets, Blazor WASM AoT, FullFramework. Cascading from Version.Details/NuGet.config changes.
#53310	Source code updates from dotnet/dotnet	release/10.0.1xx	<1d	PENDING	CI still running.

dotnet/sdk — Merge PRs (5)

#	Title	Target	Age	Checks	Issue
#53288	Merge release/10.0.3xx => main	main	<1d	PENDING	CI still running. Blocked on upstream merge chain.
#53236	Merge release/9.0.3xx => release/10.0.1xx	release/10.0.1xx	2d	PENDING	CI pending.
#53223	Merge release/10.0.2xx => release/10.0.3xx	release/10.0.3xx	3d	PENDING	CI pending — likely waiting on upstream.
#53197	Merge release/10.0.1xx => release/10.0.2xx	release/10.0.2xx	6d	PENDING	CI pending — likely waiting on upstream.
#53175 ⚠️	Merge main => release/dnup	release/dnup	7d	FAILURE	Razor, Blazor WASM, watch tests, EndToEnd, Build, Pack, Publish tests failing. Changes to eng/common infra + MSBuildSdkResolver correlate. Known issue #40006 (watch) partially applies.

dotnet/templating — Codeflow PRs (2)

#	Title	Target	Age	Checks	Issue
#9934	Source code updates from dotnet/dotnet	release/11.0.1xx-preview2	<1d	FAILURE	CI rerun triggered — templating tests can be flaky.
#9935	Source code updates from dotnet/dotnet	release/10.0.3xx	<1d	FAILURE	CI rerun triggered — templating tests can be flaky.

dotnet/dotnet (VMR) — SDK-owned PRs (6)

#	Title	Target	Age	Checks	Issue
#5268	Source code updates from dotnet/sdk	main	<1d	FAILURE	Not yet investigated.
#5177	Source code updates from dotnet/sdk	release/10.0.3xx	3d	FAILURE	Not yet investigated.
#5208	Source code updates from dotnet/source-build-reference-packages	release/10.0.1xx	2d	SUCCESS	✅ Checks pass — needs review approval only.
#5207	Source code updates from dotnet/source-build-reference-packages	release/10.0.2xx	2d	SUCCESS	✅ Checks pass — needs review approval only.
#5179	Source code updates from dotnet/sdk	release/10.0.1xx	3d	SUCCESS	✅ Checks pass — needs review approval only.
#5175	Source code updates from dotnet/templating	release/10.0.3xx	3d	SUCCESS	✅ Checks pass — needs review approval only.

dotnet/installer (0)

No open PRs from monitored authors.

📊 Summary

Category	Count
Ready to Merge	0
Branch Lockdown	0
Changes Requested	0
Failing/Blocked	17
Stale (>7d)	1
Total	17

📋 Recommended Actions

🟢 Quick wins — VMR PRs needing review only: #5208, #5207, #5179, #5175 all have passing CI but need a review approval. Approving and merging these 4 PRs would clear a chunk of the queue.
🔴 Common root cause — StaticWebAssets test failures: PRs [main] Source code updates from dotnet/dotnet #53250, [release/11.0.1xx-preview2] Source code updates from dotnet/dotnet #53267, and [release/10.0.2xx] Source code updates from dotnet/dotnet #53284 all show StaticWebAssets test failures correlated with dependency version changes flowing from dotnet/dotnet. Investigate whether a specific package version bump broke the SWA tests. Fixing this at the source will unblock multiple PRs.
🔴 Broad test failures on [release/10.0.2xx] Source code updates from dotnet/dotnet #53284 (release/10.0.2xx): 425+ test failures across NetAnalyzers, EndToEnd, Watch, etc. suggest a more fundamental dependency compatibility issue on the 10.0.2xx branch. This needs priority investigation.
🟡 Merge chain is stalled: The merge chain (9.0.3xx → 10.0.1xx → 10.0.2xx → 10.0.3xx → main) has all 5 PRs in PENDING state. These are likely blocked because upstream codeflow PRs haven't merged yet. Unblocking the codeflow PRs ([main] Source code updates from dotnet/dotnet #53250, [release/11.0.1xx-preview2] Source code updates from dotnet/dotnet #53267, [release/10.0.2xx] Source code updates from dotnet/dotnet #53284) should unstall the chain.
⚠️ Stale PR [automated] Merge branch 'main' => 'release/dnup' #53175 (7 days): Merge main → release/dnup has widespread failures (Razor, Blazor, watch, EndToEnd). Known issue dotnet-watch tests failing #40006 partially applies. Investigate whether this branch is still actively maintained — if not, consider closing this PR.
🟡 Templating PRs (Update the developer guide following changes introduced by Arcade. #9934, dotnet test --blame does not always find the right test to blame #9935): CI reruns triggered. If they pass on retry, merge them. If they fail again, investigate further.

Copilot

Pull request overview

Adds a new .claude “build-duty” skill intended to help build duty engineers triage automated PRs across key dotnet repos using deterministic gh CLI queries plus a structured JSON summary for downstream reporting.

Changes:

Introduces Get-BuildDutyReport.ps1 to query monitored PR authors across dotnet/sdk, dotnet/installer, dotnet/templating, and dotnet/dotnet, then classify and emit tables + a JSON summary block.
Adds SKILL.md documenting how to run the script and how to turn results into a build duty triage report (including using ci-analysis for failures).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 12 comments.

File	Description
.claude/skills/build-duty/scripts/Get-BuildDutyReport.ps1	New PowerShell script that queries PRs via `gh`, classifies them, prints human-readable tables, and emits a structured JSON summary.
.claude/skills/build-duty/SKILL.md	New skill documentation defining intended classification rules and the recommended triage workflow.

Copilot · 2026-03-06T18:34:47Z

.claude/skills/build-duty/scripts/Get-BuildDutyReport.ps1

+    $variables = @{
+        owner  = $Owner
+        repo   = $Repo
+        number = $Number
+    } | ConvertTo-Json -Compress
+


$variables is computed but never used; the GraphQL call passes variables via --field instead. This is dead code and can be removed, or alternatively switch to using the JSON variables payload consistently to avoid confusion about which mechanism is authoritative.

Suggested change

$variables = @{

owner = $Owner

repo = $Repo

number = $Number

} | ConvertTo-Json -Compress

Copilot · 2026-03-06T18:34:48Z

.claude/skills/build-duty/scripts/Get-BuildDutyReport.ps1

+    # Ready to merge: CLEAN merge state and SUCCESS checks
+    if ($mergeState -eq 'CLEAN' -and $checkState -eq 'SUCCESS') {
+        return 'ready'
+    }
+
+    # Also ready if merge state is CLEAN (checks may not be required)
+    if ($mergeState -eq 'CLEAN') {
+        return 'ready'
+    }
+
+    # Unstable: non-required checks failing
+    if ($mergeState -eq 'UNSTABLE') {
+        return 'ready'  # Non-required checks failing is still mergeable
+    }


Get-PrCategory returns ready for mergeStateStatus == CLEAN regardless of statusCheckRollup (including FAILURE/PENDING). This disagrees with the skill doc’s “Failing/Blocked” definition and can cause PRs with failing CI to be reported as ready. Consider incorporating checkState into the classification (or updating the doc to match the intended behavior).

Copilot · 2026-03-06T18:34:48Z

.claude/skills/build-duty/scripts/Get-BuildDutyReport.ps1

+      baseRefName
+      mergeStateStatus
+      reviewDecision
+      labels(first: 20) { nodes { name } }


The GraphQL query only fetches the first 20 labels. In these repos it’s possible for PRs to accumulate >20 labels, which could omit Branch Lockdown/DO NOT MERGE and lead to incorrect categorization. Consider increasing the page size (e.g., 100) or explicitly paging until the relevant labels are found.

Suggested change

labels(first: 20) { nodes { name } }

labels(first: 100) { nodes { name } }

Copilot · 2026-03-06T18:34:48Z

.claude/skills/build-duty/scripts/Get-BuildDutyReport.ps1

+    Write-Host ""
+    Write-Host "========================================" -ForegroundColor Cyan
+    Write-Host " Build Duty PR Triage Report" -ForegroundColor Cyan
+    Write-Host " Generated: $(Get-Date -Format 'yyyy-MM-dd HH:mm:ss UTC' -AsUTC)" -ForegroundColor Cyan


Get-Date -AsUTC is not consistently available across PowerShell versions/environments, and this script already runs cross-platform. Using [DateTimeOffset]::UtcNow (or Get-Date + ToUniversalTime()) would be more portable while still emitting a UTC timestamp.

Suggested change

Write-Host " Generated: $(Get-Date -Format 'yyyy-MM-dd HH:mm:ss UTC' -AsUTC)" -ForegroundColor Cyan

Write-Host " Generated: $([DateTimeOffset]::UtcNow.ToString('yyyy-MM-dd HH:mm:ss')) UTC" -ForegroundColor Cyan

Copilot · 2026-03-06T18:34:48Z

.claude/skills/build-duty/SKILL.md

+- Or `statusCheckRollup` is `FAILURE` or `PENDING`
+- Or has `DO NOT MERGE` label
+


The “Failing / Blocked” section says PRs are blocked when statusCheckRollup is FAILURE or PENDING, but the script’s Get-PrCategory currently can classify mergeStateStatus == CLEAN PRs as ready even when checks are failing/pending. Update this section (or the script) so the documented classification rules match the actual implementation.

Suggested change

- Or `statusCheckRollup` is `FAILURE` or `PENDING`

- Or has `DO NOT MERGE` label

- Or has `DO NOT MERGE` label

> Note: Classification here relies on GitHub's `mergeStateStatus` and labels. Individual failing or pending status checks may not always cause a PR to be categorized as Failing / Blocked if GitHub still reports `mergeStateStatus` as `CLEAN`.

Copilot · 2026-03-06T18:34:49Z

.claude/skills/build-duty/scripts/Get-BuildDutyReport.ps1

+    [int]$DaysStale = 7,
+
+    [switch]$OutputJson,
+
+    [switch]$SkipCIDetails
+)


-SkipCIDetails is declared and documented, but the switch is never used anywhere in the script. Either implement the intended behavior (skip fetching/printing CI rollup state for blocked PRs) or remove the parameter and its help text to avoid misleading users.

Copilot · 2026-03-06T18:34:49Z

.claude/skills/build-duty/scripts/Get-BuildDutyReport.ps1

+        foreach ($pr in ($ready | Sort-Object repo, ageDays -Descending)) {
+            $title = if ($pr.title.Length -gt 57) { $pr.title.Substring(0, 57) + "..." } else { $pr.title }


Sort-Object repo, ageDays -Descending applies -Descending to all sort keys, so repo will also be sorted descending. If you intended repo ascending with age descending (or similar), use per-key sort definitions (calculated properties with Descending=$true/$false).

Copilot · 2026-03-06T18:34:50Z

.claude/skills/build-duty/scripts/Get-BuildDutyReport.ps1

+        foreach ($pr in ($lockdown | Sort-Object repo, ageDays -Descending)) {
+            $title = if ($pr.title.Length -gt 57) { $pr.title.Substring(0, 57) + "..." } else { $pr.title }


Same sorting issue here: Sort-Object repo, ageDays -Descending sorts both repo and ageDays descending. Use per-key sort direction if the output is meant to be stable/predictable across repos.

Copilot · 2026-03-06T18:34:50Z

.claude/skills/build-duty/scripts/Get-BuildDutyReport.ps1

+        foreach ($pr in ($blocked | Sort-Object repo, ageDays -Descending)) {
+            $title = if ($pr.title.Length -gt 57) { $pr.title.Substring(0, 57) + "..." } else { $pr.title }


Same sorting issue here: Sort-Object repo, ageDays -Descending makes repo descending too. If repo ordering matters, use per-key sort direction.

Copilot · 2026-03-06T18:34:50Z

.claude/skills/build-duty/scripts/Get-BuildDutyReport.ps1

+        foreach ($pr in ($draft | Sort-Object repo, ageDays -Descending)) {
+            $title = if ($pr.title.Length -gt 57) { $pr.title.Substring(0, 57) + "..." } else { $pr.title }


Same sorting issue here: Sort-Object repo, ageDays -Descending sorts all keys descending. Use explicit per-key sort direction if you want repo ascending with age descending.

…y recommendations - Detect PRs with 0 changed files and recommend closing (CLOSE_EMPTY_PR) - Detect merge PRs with CONFLICTING mergeable state (FIX_MERGE_CONFLICTS) - Detect templating PRs with single failed CI leg (RETRY_SINGLE_LEG) - Add per-check-run details to GraphQL query (name, conclusion, status) - Add Quick Actions section to human-readable output - Add recommendation column to Failing/Blocked table - Update SKILL.md with recommendation codes documentation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings March 6, 2026 18:30

github-actions bot added the Area-Infrastructure label Mar 6, 2026

Copilot started reviewing on behalf of marcpopMSFT March 6, 2026 18:31 View session

Copilot AI reviewed Mar 6, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add build-duty skill for PR triage across dotnet repos#53311

Add build-duty skill for PR triage across dotnet repos#53311
marcpopMSFT wants to merge 2 commits intomainfrom
marcpopMSFT-buildreport

marcpopMSFT commented Mar 6, 2026

Uh oh!

marcpopMSFT commented Mar 6, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 6, 2026

Uh oh!

Copilot AI Mar 6, 2026

Uh oh!

Copilot AI Mar 6, 2026

Uh oh!

Copilot AI Mar 6, 2026

Uh oh!

Copilot AI Mar 6, 2026

Uh oh!

Copilot AI Mar 6, 2026

Uh oh!

Copilot AI Mar 6, 2026

Uh oh!

Copilot AI Mar 6, 2026

Uh oh!

Copilot AI Mar 6, 2026

Uh oh!

Copilot AI Mar 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	labels(first: 20) { nodes { name } }
	labels(first: 100) { nodes { name } }

	Write-Host " Generated: $(Get-Date -Format 'yyyy-MM-dd HH:mm:ss UTC' -AsUTC)" -ForegroundColor Cyan
	Write-Host " Generated: $([DateTimeOffset]::UtcNow.ToString('yyyy-MM-dd HH:mm:ss')) UTC" -ForegroundColor Cyan

		- Or `statusCheckRollup` is `FAILURE` or `PENDING`
		- Or has `DO NOT MERGE` label

		foreach ($pr in ($ready \| Sort-Object repo, ageDays -Descending)) {
		$title = if ($pr.title.Length -gt 57) { $pr.title.Substring(0, 57) + "..." } else { $pr.title }

		foreach ($pr in ($lockdown \| Sort-Object repo, ageDays -Descending)) {
		$title = if ($pr.title.Length -gt 57) { $pr.title.Substring(0, 57) + "..." } else { $pr.title }

		foreach ($pr in ($blocked \| Sort-Object repo, ageDays -Descending)) {
		$title = if ($pr.title.Length -gt 57) { $pr.title.Substring(0, 57) + "..." } else { $pr.title }

		foreach ($pr in ($draft \| Sort-Object repo, ageDays -Descending)) {
		$title = if ($pr.title.Length -gt 57) { $pr.title.Substring(0, 57) + "..." } else { $pr.title }

Conversation

marcpopMSFT commented Mar 6, 2026

Uh oh!

marcpopMSFT commented Mar 6, 2026

🔧 Build Duty Triage Report

✅ Ready to Merge (0)

🔒 Branch Lockdown (0)

⚠️ Changes Requested (0)

❌ Failing / Blocked (17)

dotnet/sdk — Codeflow PRs (4)

dotnet/sdk — Merge PRs (5)

dotnet/templating — Codeflow PRs (2)

dotnet/dotnet (VMR) — SDK-owned PRs (6)

dotnet/installer (0)

📊 Summary

📋 Recommended Actions

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants