Skip to content

Commit 022d89a

Browse files
danmoseleyCopilot
andauthored
Initialize repository with checkin validation and ci-analysis skill (#2)
* Initialize repository infrastructure Add checkin validation modeled on dotnet/skills (without AI evaluation): - CODEOWNERS validation workflow (from dotnet/skills, verbatim) - Structural PR validation workflow: plugin.json, SKILL.md frontmatter, eval.yaml schema, marketplace.json consistency, orphaned test detection - Plugin marketplace configuration (.github/plugin + .claude-plugin) - Dependabot for GitHub Actions updates - Repository docs: README, CONTRIBUTING, AGENTS, LICENSE - Root configs: global.json (.NET 10 SDK), .gitattributes Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add comprehensive structural validation checks Adds all non-AI validation from dotnet/skills' skill-validator: - Path traversal protection on skills/agents paths in plugin.json - Compatibility field validation (<=500 chars) - Body line count limit (<=500 lines) - Token size analysis (chars/4 approximation) with complexity warnings - File reference depth check (max 1 dir deep, no parent traversal) - Structural warnings (sections, code blocks, numbered steps) - Eval prompt bias detection (skill name in scenario prompt) - Per-assertion required fields (path, value, pattern per type) - Unknown assertion type detection Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add ci-analysis skill from lewing/agent-plugins Adds the dotnet-dnceng plugin with the ci-analysis skill, synced from lewing/agent-plugins. This skill analyzes CI build status and test failures in Azure DevOps and Helix for dotnet repositories. Includes: - SKILL.md with full workflow documentation - Get-CIStatus.ps1 script (PR, build, and Helix analysis modes) - 12 reference docs (analysis workflow, failure interpretation, binlog comparison, build progression, delegation patterns, etc.) - plugin.json with MCP server configurations - Updated marketplace.json to register the plugin This enables dotnet/runtime and other repos to reference ci-analysis from dotnet/arcade-skills instead of maintaining local copies. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * TEST: introduce deliberate spec violations (will revert) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add CODEOWNERS entries, revert test violation Adds @lewing and @danmoseley as code owners for the ci-analysis skill, workflows, and repo default. Reverts the deliberate SKILL.md frontmatter violation used to verify pr-validation catches errors. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add @ViktorHofer to CODEOWNERS Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Post PR comment with validation errors When structural validation finds errors or warnings, posts a comment on the PR listing them (updates existing comment on re-push). Uses a marker comment for idempotent updates. Follows the same pattern as dotnet/skills' CODEOWNERS validation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address Copilot review feedback - URL-encode WorkItem in Helix console URL to handle special chars - Use versioned Helix API URL consistently (2019-06-17) - Update README plugin table to list dotnet-dnceng - (PR description updated separately to reflect CODEOWNERS state) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 8f2643e commit 022d89a

29 files changed

+5008
-0
lines changed

.claude-plugin/marketplace.json

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"name": "dotnet-arcade-skills",
3+
"owner": {
4+
"name": ".NET Team at Microsoft"
5+
},
6+
"plugins": [
7+
{
8+
"name": "dotnet-dnceng",
9+
"source": "./plugins/dotnet-dnceng",
10+
"description": "Skills for .NET engineering infrastructure: CI/CD analysis, build pipeline workflows",
11+
"version": "0.1.0"
12+
}
13+
]
14+
}

.gitattributes

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
* text=auto eol=lf

.github/CODEOWNERS

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# Default owners for everything in the repo
2+
* @lewing @danmoseley @viktorhofer
3+
4+
# CI/CD and engineering
5+
/.github/workflows/ @lewing @danmoseley @viktorhofer
6+
7+
# Skills
8+
/plugins/dotnet-dnceng/skills/ci-analysis/ @lewing @danmoseley

.github/dependabot.yml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
version: 2
2+
updates:
3+
- package-ecosystem: github-actions
4+
directory: /
5+
schedule:
6+
interval: weekly
7+
open-pull-requests-limit: 5
8+
groups:
9+
github-actions-dependencies:
10+
patterns:
11+
- "*"

.github/plugin/marketplace.json

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"name": "dotnet-arcade-skills",
3+
"owner": {
4+
"name": ".NET Team at Microsoft"
5+
},
6+
"plugins": [
7+
{
8+
"name": "dotnet-dnceng",
9+
"source": "./plugins/dotnet-dnceng",
10+
"description": "Skills for .NET engineering infrastructure: CI/CD analysis, build pipeline workflows",
11+
"version": "0.1.0"
12+
}
13+
]
14+
}
Lines changed: 256 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,256 @@
1+
name: codeowners-folder-validation
2+
3+
on:
4+
pull_request:
5+
push:
6+
branches: [main]
7+
paths:
8+
- 'plugins/**'
9+
- 'tests/**'
10+
- '.github/CODEOWNERS'
11+
- '.github/workflows/codeowners-folder-validation.yml'
12+
workflow_dispatch:
13+
14+
permissions:
15+
contents: read
16+
issues: write
17+
18+
jobs:
19+
validate-codeowners:
20+
runs-on: ubuntu-latest
21+
22+
steps:
23+
- name: Checkout repository
24+
uses: actions/checkout@v6
25+
26+
- name: Find missing CODEOWNERS folder entries
27+
id: audit
28+
shell: pwsh
29+
run: |
30+
$codeownersPath = '.github/CODEOWNERS'
31+
if (-not (Test-Path $codeownersPath)) {
32+
Write-Error "Missing $codeownersPath"
33+
exit 1
34+
}
35+
36+
# Collect CODEOWNERS path tokens that represent concrete directory scopes.
37+
# Wildcard tokens are ignored for this validation to keep checks folder-specific.
38+
$codeownerEntries = New-Object 'System.Collections.Generic.List[PSCustomObject]'
39+
$ownedPaths = New-Object 'System.Collections.Generic.List[string]'
40+
$lines = Get-Content -Path $codeownersPath
41+
foreach ($line in $lines) {
42+
$trimmed = $line.Trim()
43+
if ([string]::IsNullOrWhiteSpace($trimmed) -or $trimmed.StartsWith('#')) {
44+
continue
45+
}
46+
47+
$tokens = $trimmed -split '\s+'
48+
$pathToken = $tokens[0]
49+
if ($pathToken -match '[*?\[]') {
50+
continue
51+
}
52+
if (-not $pathToken.StartsWith('/')) {
53+
$pathToken = "/$pathToken"
54+
}
55+
if (-not $pathToken.EndsWith('/')) {
56+
$pathToken = "$pathToken/"
57+
}
58+
59+
$owners = @($tokens | Select-Object -Skip 1 | Where-Object { $_ -match '^@' })
60+
$codeownerEntries.Add([PSCustomObject]@{
61+
Path = $pathToken
62+
Owners = $owners
63+
})
64+
$ownedPaths.Add($pathToken)
65+
}
66+
67+
$expectedPaths = New-Object 'System.Collections.Generic.HashSet[string]' ([System.StringComparer]::OrdinalIgnoreCase)
68+
69+
if (Test-Path 'plugins') {
70+
$pluginDirs = Get-ChildItem -Path 'plugins' -Directory
71+
foreach ($pluginDir in $pluginDirs) {
72+
$skillsRoot = Join-Path $pluginDir.FullName 'skills'
73+
if (-not (Test-Path $skillsRoot)) {
74+
continue
75+
}
76+
77+
$skillDirs = Get-ChildItem -Path $skillsRoot -Directory
78+
foreach ($skillDir in $skillDirs) {
79+
[void]$expectedPaths.Add("/plugins/$($pluginDir.Name)/skills/$($skillDir.Name)/")
80+
}
81+
}
82+
}
83+
84+
if (Test-Path 'tests') {
85+
$testPluginDirs = Get-ChildItem -Path 'tests' -Directory
86+
foreach ($testPluginDir in $testPluginDirs) {
87+
$testSkillDirs = Get-ChildItem -Path $testPluginDir.FullName -Directory
88+
foreach ($testSkillDir in $testSkillDirs) {
89+
[void]$expectedPaths.Add("/tests/$($testPluginDir.Name)/$($testSkillDir.Name)/")
90+
}
91+
}
92+
}
93+
94+
function Test-IsCovered([string]$ExpectedPath, [System.Collections.Generic.List[string]]$Scopes) {
95+
foreach ($scope in $Scopes) {
96+
if ($ExpectedPath.StartsWith($scope, [System.StringComparison]::OrdinalIgnoreCase)) {
97+
return $true
98+
}
99+
}
100+
return $false
101+
}
102+
103+
# Return the last matching CODEOWNERS entry (last-match-wins semantics).
104+
function Get-EffectiveEntry([string]$ExpectedPath, [System.Collections.Generic.List[PSCustomObject]]$Entries) {
105+
$lastMatch = $null
106+
foreach ($entry in $Entries) {
107+
if ($ExpectedPath.StartsWith($entry.Path, [System.StringComparison]::OrdinalIgnoreCase)) {
108+
$lastMatch = $entry
109+
}
110+
}
111+
return $lastMatch
112+
}
113+
114+
# Require at least one team or at least two individuals.
115+
function Test-SufficientOwners([string[]]$Owners) {
116+
$teams = @($Owners | Where-Object { $_ -match '/' })
117+
if ($teams.Count -ge 1) { return $true }
118+
$individuals = @($Owners | Where-Object { $_ -notmatch '/' })
119+
if ($individuals.Count -ge 2) { return $true }
120+
return $false
121+
}
122+
123+
# --- Check 1: missing CODEOWNERS entries ---
124+
$missing = @(
125+
$expectedPaths |
126+
Where-Object { -not (Test-IsCovered $_ $ownedPaths) } |
127+
Sort-Object
128+
)
129+
130+
# --- Check 2: insufficient owners (only for covered paths) ---
131+
$insufficientOwners = @(
132+
$expectedPaths |
133+
Where-Object { Test-IsCovered $_ $ownedPaths } |
134+
ForEach-Object {
135+
$entry = Get-EffectiveEntry $_ $codeownerEntries
136+
if ($entry -and -not (Test-SufficientOwners $entry.Owners)) {
137+
[PSCustomObject]@{
138+
path = $_
139+
owners = $entry.Owners -join ' '
140+
}
141+
}
142+
} |
143+
Sort-Object -Property path
144+
)
145+
146+
# --- Outputs ---
147+
$missingJson = ConvertTo-Json -InputObject @($missing) -Compress
148+
if (-not $missingJson) { $missingJson = '[]' }
149+
150+
$insufficientJson = ConvertTo-Json -InputObject @($insufficientOwners) -Compress -Depth 3
151+
if (-not $insufficientJson) { $insufficientJson = '[]' }
152+
153+
$hasMissing = $missing.Count -gt 0
154+
$hasInsufficient = $insufficientOwners.Count -gt 0
155+
156+
if ($hasMissing) {
157+
Write-Host "Missing CODEOWNERS entries:"
158+
$missing | ForEach-Object { Write-Host " - $_" }
159+
}
160+
if ($hasInsufficient) {
161+
Write-Host "Insufficient owners (need 2+ individuals or 1+ team):"
162+
$insufficientOwners | ForEach-Object { Write-Host " - $($_.path) (current: $($_.owners))" }
163+
}
164+
if (-not $hasMissing -and -not $hasInsufficient) {
165+
Write-Host 'All CODEOWNERS entries are present and have sufficient owners.'
166+
}
167+
168+
"has_missing=$($hasMissing.ToString().ToLower())" >> $env:GITHUB_OUTPUT
169+
"missing_json=$missingJson" >> $env:GITHUB_OUTPUT
170+
"has_insufficient_owners=$($hasInsufficient.ToString().ToLower())" >> $env:GITHUB_OUTPUT
171+
"insufficient_owners_json=$insufficientJson" >> $env:GITHUB_OUTPUT
172+
173+
- name: Create or update issue for validation failures
174+
if: (steps.audit.outputs.has_missing == 'true' || steps.audit.outputs.has_insufficient_owners == 'true') && github.event_name != 'pull_request'
175+
uses: actions/github-script@v8
176+
env:
177+
MISSING_JSON: ${{ steps.audit.outputs.missing_json }}
178+
INSUFFICIENT_OWNERS_JSON: ${{ steps.audit.outputs.insufficient_owners_json }}
179+
with:
180+
script: |
181+
const missing = JSON.parse(process.env.MISSING_JSON || '[]');
182+
const insufficientOwners = JSON.parse(process.env.INSUFFICIENT_OWNERS_JSON || '[]');
183+
184+
if ((!Array.isArray(missing) || missing.length === 0) &&
185+
(!Array.isArray(insufficientOwners) || insufficientOwners.length === 0)) {
186+
core.info('No issues to report.');
187+
return;
188+
}
189+
190+
const title = 'CODEOWNERS validation failures for skill/test folders';
191+
const marker = '<!-- codeowners-folder-validation -->';
192+
const runUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
193+
const bodyParts = [
194+
marker,
195+
'Issues discovered by workflow `codeowners-folder-validation`.',
196+
''
197+
];
198+
199+
if (missing.length > 0) {
200+
bodyParts.push('## Missing CODEOWNERS entries', '');
201+
bodyParts.push(...missing.map((p) => `- \`${p}\``));
202+
bodyParts.push('');
203+
}
204+
205+
if (insufficientOwners.length > 0) {
206+
bodyParts.push('## Insufficient owners', '');
207+
bodyParts.push('Each skill/test folder must have at least **2 individual owners** or **1 team**.', '');
208+
bodyParts.push(...insufficientOwners.map((e) => `- \`${e.path}\` — current: ${e.owners}`));
209+
bodyParts.push('');
210+
}
211+
212+
bodyParts.push(`Run: ${runUrl}`);
213+
const body = bodyParts.join('\n');
214+
215+
const { data: issues } = await github.rest.issues.listForRepo({
216+
owner: context.repo.owner,
217+
repo: context.repo.repo,
218+
state: 'open',
219+
per_page: 100
220+
});
221+
222+
const existing = issues.find((i) => i.title === title);
223+
224+
if (existing) {
225+
await github.rest.issues.update({
226+
owner: context.repo.owner,
227+
repo: context.repo.repo,
228+
issue_number: existing.number,
229+
body
230+
});
231+
core.info(`Updated issue #${existing.number}`);
232+
} else {
233+
const created = await github.rest.issues.create({
234+
owner: context.repo.owner,
235+
repo: context.repo.repo,
236+
title,
237+
body
238+
});
239+
core.info(`Created issue #${created.data.number}`);
240+
}
241+
242+
- name: Fail when CODEOWNERS validation fails
243+
if: steps.audit.outputs.has_missing == 'true' || steps.audit.outputs.has_insufficient_owners == 'true'
244+
shell: pwsh
245+
env:
246+
MISSING_JSON: ${{ steps.audit.outputs.missing_json }}
247+
INSUFFICIENT_OWNERS_JSON: ${{ steps.audit.outputs.insufficient_owners_json }}
248+
run: |
249+
if ($env:MISSING_JSON -ne '[]') {
250+
Write-Host "Missing entries: $env:MISSING_JSON"
251+
}
252+
if ($env:INSUFFICIENT_OWNERS_JSON -ne '[]') {
253+
Write-Host "Insufficient owners: $env:INSUFFICIENT_OWNERS_JSON"
254+
}
255+
Write-Error 'CODEOWNERS validation failed. Each skill/test folder needs a CODEOWNERS entry with 2+ individuals or 1+ team.'
256+
exit 1

0 commit comments

Comments
 (0)