fix(scripts): eliminate phantom git changes from plugin generation (#1035)

## Description

Eliminated phantom git changes that `npm run plugin:generate` left in
the working tree after every run. The root cause was a
delete-and-recreate strategy that touched file timestamps and re-wrote
identical content, combined with a `--index-info` stdin call that broke
on Windows due to CRLF encoding.

### Plugin Generation — Conditional Writes

Introduced `Set-ContentIfChanged` in *CollectionHelpers.psm1* as the
single write choke point for all generated files. The function performs
an ordinal comparison of proposed content against the existing file and
skips the write when identical. Converted all five production write
sites in *PluginHelpers.psm1* — `Write-MarketplaceManifest`,
`New-PluginLink` fallback, *plugin.json*, *README.md*, and collection
YAML — to route through `Set-ContentIfChanged`.

Added a **GeneratedFiles** `HashSet` to `Write-PluginDirectory` that
tracks every output path and returns it in the result object, enabling
downstream orphan detection.

Rewrote `Repair-PluginSymlinkIndex` to parse `git ls-files --stage`
output, detect mode-120000 entries, skip already-correct symlinks, and
use per-entry `--cacheinfo` calls instead of piped `--index-info`. The
piped approach broke on Windows because PowerShell injected CRLF into
stdin, corrupting the index update.

### Plugin Generation — Orphan Cleanup

Replaced the delete-and-recreate Refresh strategy in
*Generate-Plugins.ps1* with orphan cleanup. After overwrite-in-place
generation completes, files not present in the `GeneratedFiles` set are
removed, followed by a bottom-up empty-directory sweep. All
`Remove-Item` calls gained `-ErrorAction Stop` for fail-fast behavior.

### Post-Processing Scope

Extracted post-processing into a `plugin:postprocess` script in
*package.json*, scoping *markdownlint-cli2* and
*markdown-table-formatter* to `plugins/**/*.md` and `collections/*.md`
only. This prevented post-processing from modifying unrelated files in
the working tree.

### Windows Test Compatibility

Separately fixed 14 pre-existing test failures in
*Invoke-PptxPipeline.Tests.ps1*. The tests created shell-script `.exe`
python stubs that are not valid PE executables on Windows. Replaced with
`.cmd` batch files that execute natively and correctly set
`$LASTEXITCODE`.

### Test Coverage

- Added a **Set-ContentIfChanged** test suite in
*CollectionHelpers.Tests.ps1* covering creation, skip-on-identical,
overwrite-on-diff, case-sensitivity, empty string, and UTF-8-without-BOM
behavior.
- Added 4 **Orphan Cleanup** tests in *Generate-Plugins.Tests.ps1*
validating orphan removal, generated file preservation, empty directory
pruning, and DryRun logging.
- Added 5 **Repair-PluginSymlinkIndex** tests in
*PluginHelpers.Tests.ps1* using a real git repo fixture — covers DryRun
counting, DryRun index immutability, mode-120000 re-indexing, and
skip-if-already-fixed logic.

## Related Issue(s)

Closes #1029

## Type of Change

Select all that apply:

**Code & Documentation:**

* [x] Bug fix (non-breaking change fixing an issue)
* [ ] New feature (non-breaking change adding functionality)
* [ ] Breaking change (fix or feature causing existing functionality to
change)
* [ ] Documentation update

**Infrastructure & Configuration:**

* [ ] GitHub Actions workflow
* [ ] Linting configuration (markdown, PowerShell, etc.)
* [ ] Security configuration
* [ ] DevContainer configuration
* [ ] Dependency update

**AI Artifacts:**

* [ ] Reviewed contribution with `prompt-builder` agent and addressed
all feedback
* [ ] Copilot instructions (`.github/instructions/*.instructions.md`)
* [ ] Copilot prompt (`.github/prompts/*.prompt.md`)
* [ ] Copilot agent (`.github/agents/*.agent.md`)
* [ ] Copilot skill (`.github/skills/*/SKILL.md`)

**Other:**

* [x] Script/automation (`.ps1`, `.sh`, `.py`)
* [ ] Other (please describe):

## Testing

Full test suite validated after both commits:

- **1707 passed**, 0 failed, 1 skipped
- PSScriptAnalyzer clean on all modified PowerShell files
- New tests cover `Set-ContentIfChanged`, orphan cleanup, and
`Repair-PluginSymlinkIndex` comprehensively

## Checklist

### Required Checks

* [ ] Documentation is updated (if applicable)
* [x] Files follow existing naming conventions
* [x] Changes are backwards compatible (if applicable)
* [x] Tests added for new functionality (if applicable)

### AI Artifact Contributions
<!-- Not applicable — no AI artifacts modified -->

### Required Automated Checks

The following validation commands must pass before merging:

* [x] Markdown linting: `npm run lint:md`
* [x] Spell checking: `npm run spell-check`
* [x] Frontmatter validation: `npm run lint:frontmatter`
* [x] Skill structure validation: `npm run validate:skills`
* [x] Link validation: `npm run lint:md-links` (pre-existing failures
only — no new broken links introduced)
* [x] PowerShell analysis: `npm run lint:ps`
* [x] Plugin freshness: `npm run plugin:generate`

## Security Considerations

* [x] This PR does not contain any sensitive or NDA information
* [x] Any new dependencies have been reviewed for security issues
* [x] Security-related scripts follow the principle of least privilege

## Additional Notes

- Two test files (*CollectionHelpers.Tests.ps1*,
*PluginHelpers.Tests.ps1*) gained UTF-8 BOM byte prefix — incidental
encoding change from editor saves on Windows, no functional impact.
- No new dependencies introduced. All changes are internal to existing
PowerShell modules and npm scripts.

---------

Co-authored-by: Bill Berry <wbery@microsoft.com>

Author: WilliamBerryiii

package.json

@@ -30,7 +30,8 @@
     "validate:skills": "pwsh -NoProfile -Command \"& './scripts/linting/Validate-SkillStructure.ps1' -WarningsAsErrors\"",
     "lint:py": "pwsh -NoProfile -File ./scripts/linting/Invoke-PythonLint.ps1",
     "test:py": "pwsh -NoProfile -File ./scripts/linting/Invoke-PythonTests.ps1",
-    "plugin:generate": "pwsh -File scripts/plugins/Generate-Plugins.ps1 && npm run lint:md:fix && npm run format:tables",
+    "plugin:postprocess": "markdownlint-cli2 \"plugins/**/*.md\" \"collections/*.md\" --fix && markdown-table-formatter \"plugins/**/*.md\" && markdown-table-formatter \"collections/*.md\"",
+    "plugin:generate": "pwsh -File scripts/plugins/Generate-Plugins.ps1 && npm run plugin:postprocess",
     "plugin:validate": "npm run lint:collections-metadata",
     "docs:build": "npm --prefix docs/docusaurus run build",
     "docs:serve": "npm --prefix docs/docusaurus run serve"

scripts/collections/Modules/CollectionHelpers.psm1

@@ -9,6 +9,50 @@
 #Requires -Version 7.0
 #Requires -Modules PowerShell-Yaml
 
+# ---------------------------------------------------------------------------
+# Internal Utilities
+# ---------------------------------------------------------------------------
+
+function Set-ContentIfChanged {
+    <#
+    .SYNOPSIS
+        Writes content to a file only when the content has changed.
+    .DESCRIPTION
+        Compares the provided value against the existing file content using
+        case-sensitive ordinal comparison. Writes only when the file does not
+        exist or content differs, preserving the git stat cache for unchanged files.
+    .PARAMETER Path
+        The file path to write.
+    .PARAMETER Value
+        The content to write.
+    .OUTPUTS
+        [bool] True if the file was written, false if skipped.
+    #>
+    [CmdletBinding()]
+    [OutputType([bool])]
+    param(
+        [Parameter(Mandatory)]
+        [string]$Path,
+
+        [Parameter(Mandatory)]
+        [AllowEmptyString()]
+        [string]$Value
+    )
+
+    if (Test-Path -LiteralPath $Path) {
+        $existing = Get-Content -LiteralPath $Path -Raw -Encoding utf8
+        if ([string]::Equals($existing, $Value, [System.StringComparison]::Ordinal)) {
+            return $false
+        }
+    }
+    $parentDir = Split-Path -Path $Path -Parent
+    if ($parentDir -and -not (Test-Path -LiteralPath $parentDir)) {
+        New-Item -ItemType Directory -Path $parentDir -Force | Out-Null
+    }
+    Set-Content -LiteralPath $Path -Value $Value -Encoding utf8 -NoNewline
+    return $true
+}
+
 # ---------------------------------------------------------------------------
 # Pure Functions (no file system side effects)
 # ---------------------------------------------------------------------------
@@ -538,7 +582,7 @@ function Update-HveCoreAllCollection {
         }
 
         $yaml = ConvertTo-Yaml -Data $manifest
-        Set-Content -Path $collectionPath -Value $yaml -Encoding utf8 -NoNewline
+        Set-ContentIfChanged -Path $collectionPath -Value $yaml | Out-Null
         Write-Verbose "Updated $collectionPath"
     }
 
@@ -557,6 +601,7 @@ Export-ModuleMember -Function @(
     'Get-CollectionArtifactKey',
     'Get-CollectionManifest',
     'Resolve-CollectionItemMaturity',
+    'Set-ContentIfChanged',
     'Test-ArtifactDeprecated',
     'Test-DeprecatedPath',
     'Test-HveCoreRepoRelativePath',

scripts/plugins/Generate-Plugins.ps1

@@ -255,18 +255,7 @@ function Invoke-PluginGeneration {
             continue
         }
 
-        # Refresh: remove existing plugin directory
-        if ($Refresh -and (Test-Path -Path $pluginDir)) {
-            if ($DryRun) {
-                Write-Host "  [DRY RUN] Would remove $pluginDir" -ForegroundColor Yellow
-            }
-            else {
-                Remove-Item -Path $pluginDir -Recurse -Force
-                Write-Verbose "Removed existing plugin directory: $pluginDir"
-            }
-        }
-
-        # Generate plugin directory structure
+        # Generate plugin directory structure (overwrites in place)
         $filteredCollection = Select-CollectionItemsByChannel -Collection $collection -Channel $Channel
 
         $result = Write-PluginDirectory -Collection $filteredCollection `
@@ -277,6 +266,47 @@ function Invoke-PluginGeneration {
             -DryRun:$DryRun `
             -SymlinkCapable:$symlinkCapable
 
+        # Orphan cleanup in Refresh mode
+        if ($Refresh -and (Test-Path -LiteralPath $pluginDir)) {
+            $generatedFiles = $result.GeneratedFiles
+            $existingFiles = [System.Collections.Generic.List[string]]::new()
+            $scanQueue = [System.Collections.Generic.Queue[string]]::new()
+            $scanQueue.Enqueue($pluginDir)
+            while ($scanQueue.Count -gt 0) {
+                $currentDir = $scanQueue.Dequeue()
+                foreach ($entry in Get-ChildItem -LiteralPath $currentDir -Force) {
+                    if ($entry.PSIsContainer -and -not $entry.LinkType) {
+                        $scanQueue.Enqueue($entry.FullName)
+                    }
+                    else {
+                        $existingFiles.Add($entry.FullName)
+                    }
+                }
+            }
+            foreach ($existingFile in $existingFiles) {
+                if (-not $generatedFiles.Contains($existingFile)) {
+                    if ($DryRun) {
+                        Write-Host "  [DRY RUN] Would remove orphan: $existingFile" -ForegroundColor Yellow
+                    }
+                    else {
+                        Remove-Item -LiteralPath $existingFile -Force -ErrorAction Stop
+                        Write-Verbose "Removed orphan file: $existingFile"
+                    }
+                }
+            }
+            # Remove empty directories bottom-up
+            if (-not $DryRun) {
+                Get-ChildItem -LiteralPath $pluginDir -Recurse -Directory |
+                    Where-Object { -not $_.LinkType } |
+                    Sort-Object { $_.FullName.Length } -Descending |
+                    Where-Object { @(Get-ChildItem -LiteralPath $_.FullName).Count -eq 0 } |
+                    ForEach-Object {
+                        Remove-Item -LiteralPath $_.FullName -Force -ErrorAction Stop
+                        Write-Verbose "Removed empty directory: $($_.FullName)"
+                    }
+            }
+        }
+
         $itemCount = $filteredCollection.items.Count
         $totalAgents += $result.AgentCount
         $totalCommands += $result.CommandCount

scripts/plugins/Modules/PluginHelpers.psm1

@@ -399,7 +399,8 @@ function Write-MarketplaceManifest {
         New-Item -ItemType Directory -Path $outputDir -Force | Out-Null
     }
 
-    $manifest | ConvertTo-Json -Depth 10 | Set-Content -Path $outputPath -Encoding utf8 -NoNewline
+    $manifestJson = $manifest | ConvertTo-Json -Depth 10
+    Set-ContentIfChanged -Path $outputPath -Value $manifestJson | Out-Null
     Write-Host "  Marketplace manifest: $outputPath" -ForegroundColor Green
 }
 
@@ -525,7 +526,7 @@ function New-PluginLink {
         New-Item -ItemType SymbolicLink -Path $DestinationPath -Value $relativePath -Force | Out-Null
     }
     else {
-        [System.IO.File]::WriteAllText($DestinationPath, $relativePath)
+        Set-ContentIfChanged -Path $DestinationPath -Value $relativePath | Out-Null
     }
 }
 
@@ -604,6 +605,9 @@ function Write-PluginDirectory {
     }
 
     $readmeItems = @()
+    $generatedFiles = [System.Collections.Generic.HashSet[string]]::new(
+        [System.StringComparer]::OrdinalIgnoreCase
+    )
 
     foreach ($item in $Collection.items) {
         $kind = $item.kind
@@ -657,6 +661,8 @@ function Write-PluginDirectory {
             'skill'       { $counts.SkillCount++ }
         }
 
+        [void]$generatedFiles.Add($destPath)
+
         if ($DryRun) {
             Write-Verbose "DryRun: Would create link $destPath -> $sourcePath"
             continue
@@ -680,6 +686,8 @@ function Write-PluginDirectory {
             continue
         }
 
+        [void]$generatedFiles.Add($destPath)
+
         if ($DryRun) {
             Write-Verbose "DryRun: Would create shared directory link $destPath -> $sourcePath"
             continue
@@ -692,6 +700,7 @@ function Write-PluginDirectory {
     $manifestDir = Join-Path -Path $pluginRoot -ChildPath '.github' -AdditionalChildPath 'plugin'
     $manifestPath = Join-Path -Path $manifestDir -ChildPath 'plugin.json'
     $manifest = New-PluginManifestContent -CollectionId $collectionId -Description $Collection.description -Version $Version
+    [void]$generatedFiles.Add($manifestPath)
 
     if ($DryRun) {
         Write-Verbose "DryRun: Would write plugin.json at $manifestPath"
@@ -700,7 +709,8 @@ function Write-PluginDirectory {
         if (-not (Test-Path -Path $manifestDir)) {
             New-Item -ItemType Directory -Path $manifestDir -Force | Out-Null
         }
-        $manifest | ConvertTo-Json -Depth 10 | Set-Content -Path $manifestPath -Encoding utf8 -NoNewline
+        $jsonContent = $manifest | ConvertTo-Json -Depth 10
+        Set-ContentIfChanged -Path $manifestPath -Value $jsonContent | Out-Null
     }
 
     # Generate README.md
@@ -710,12 +720,13 @@ function Write-PluginDirectory {
         Get-Content -Path $collectionMdPath -Raw
     } else { $null }
     $readmeContent = New-PluginReadmeContent -Collection $Collection -Items $readmeItems -Maturity $Maturity -CollectionContent $collectionContent
+    [void]$generatedFiles.Add($readmePath)
 
     if ($DryRun) {
         Write-Verbose "DryRun: Would write README.md at $readmePath"
     }
     else {
-        Set-Content -Path $readmePath -Value $readmeContent -Encoding utf8 -NoNewline
+        Set-ContentIfChanged -Path $readmePath -Value $readmeContent | Out-Null
     }
 
     return @{
@@ -724,6 +735,7 @@ function Write-PluginDirectory {
         CommandCount     = $counts.CommandCount
         InstructionCount = $counts.InstructionCount
         SkillCount       = $counts.SkillCount
+        GeneratedFiles   = $generatedFiles
     }
 }
 
@@ -776,15 +788,23 @@ function Repair-PluginSymlinkIndex {
     $trackedPaths = [System.Collections.Generic.HashSet[string]]::new(
         [System.StringComparer]::OrdinalIgnoreCase
     )
+    $alreadySymlink = [System.Collections.Generic.HashSet[string]]::new(
+        [System.StringComparer]::OrdinalIgnoreCase
+    )
     $pluginsRel = [System.IO.Path]::GetRelativePath($RepoRoot, $PluginsDir) -replace '\\', '/'
-    $lsOutput = git ls-files -- $pluginsRel 2>$null
+    $lsOutput = git ls-files --stage -- $pluginsRel 2>$null
     if ($lsOutput) {
-        foreach ($p in @($lsOutput)) { [void]$trackedPaths.Add($p) }
+        foreach ($line in @($lsOutput)) {
+            if ($line -match '^(\d+)\s+[0-9a-f]+\s+\d+\t(.+)$') {
+                [void]$trackedPaths.Add($Matches[2])
+                if ($Matches[1] -eq '120000') {
+                    [void]$alreadySymlink.Add($Matches[2])
+                }
+            }
+        }
     }
 
     $fixedCount = 0
-    $newEntries = [System.Collections.Generic.List[PSCustomObject]]::new()
-    $batchEntries = [System.Collections.Generic.List[string]]::new()
     $files = Get-ChildItem -Path $PluginsDir -File -Recurse
 
     foreach ($file in $files) {
@@ -805,6 +825,10 @@ function Repair-PluginSymlinkIndex {
 
         $repoRelPath = [System.IO.Path]::GetRelativePath($RepoRoot, $file.FullName) -replace '\\', '/'
 
+        if ($alreadySymlink.Contains($repoRelPath)) {
+            continue
+        }
+
         if ($DryRun) {
             Write-Verbose "DryRun: Would fix index mode for $repoRelPath"
             $fixedCount++
@@ -824,33 +848,18 @@ function Repair-PluginSymlinkIndex {
             continue
         }
 
-        if ($trackedPaths.Contains($repoRelPath)) {
-            $batchEntries.Add("120000 $sha`t$repoRelPath")
-        } else {
-            $newEntries.Add([PSCustomObject]@{ Sha = $sha; Path = $repoRelPath })
-        }
-        $fixedCount++
-        Write-Verbose "Queued index fix: $repoRelPath -> 120000"
-    }
-
-    # Add new/untracked files individually (typically few per run)
-    foreach ($entry in $newEntries) {
-        $cacheResult = git update-index --add --cacheinfo "120000,$($entry.Sha),$($entry.Path)" 2>&1
+        # Use --add for untracked files; harmless for already-tracked entries.
+        # Avoids --index-info piping which breaks on Windows due to CRLF stdin.
+        $addFlag = if (-not $trackedPaths.Contains($repoRelPath)) { '--add' } else { $null }
+        $cacheArgs = @('update-index') + @($addFlag | Where-Object { $_ }) + @('--cacheinfo', "120000,$sha,$repoRelPath")
+        $cacheResult = & git @cacheArgs 2>&1
         if ($LASTEXITCODE -ne 0) {
             $errorMsg = @($cacheResult | ForEach-Object { $_.ToString() }) -join '; '
-            Write-Warning "Failed to add index entry for $($entry.Path): $errorMsg"
-            $fixedCount--
-        }
-    }
-
-    # Batch update existing entries in a single call to avoid index.lock contention
-    if ($batchEntries.Count -gt 0) {
-        $indexResult = $batchEntries | git update-index --index-info 2>&1
-        if ($LASTEXITCODE -ne 0) {
-            $errorMsg = @($indexResult | ForEach-Object { $_.ToString() }) -join '; '
-            Write-Warning "Failed to update git index: $errorMsg"
-            return 0
+            Write-Warning "Failed to update index entry for ${repoRelPath}: $errorMsg"
+            continue
         }
+        $fixedCount++
+        Write-Verbose "Fixed index mode: $repoRelPath -> 120000"
     }
 
     return $fixedCount

scripts/tests/collections/CollectionHelpers.Tests.ps1

@@ -1,4 +1,4 @@
-#Requires -Modules Pester
+#Requires -Modules Pester
 # Copyright (c) Microsoft Corporation.
 # SPDX-License-Identifier: MIT
 
@@ -562,3 +562,63 @@ display:
         $output | Should -Match 'items: \[\]' -Because 'DryRun should not modify the file'
     }
 }
+
+Describe 'Set-ContentIfChanged' {
+    BeforeAll {
+        $script:testDir = Join-Path $TestDrive ([System.Guid]::NewGuid().ToString())
+        New-Item -ItemType Directory -Path $script:testDir -Force | Out-Null
+    }
+
+    It 'Creates file when it does not exist' {
+        $path = Join-Path $script:testDir 'new-file.txt'
+        $result = Set-ContentIfChanged -Path $path -Value 'hello'
+        $result | Should -BeTrue
+        Get-Content -Path $path -Raw -Encoding utf8 | Should -Be 'hello'
+    }
+
+    It 'Skips write when content is identical' {
+        $path = Join-Path $script:testDir 'same-content.txt'
+        Set-Content -Path $path -Value 'unchanged' -Encoding utf8 -NoNewline
+        $before = (Get-Item -LiteralPath $path).LastWriteTimeUtc
+        Start-Sleep -Milliseconds 50
+        $result = Set-ContentIfChanged -Path $path -Value 'unchanged'
+        $result | Should -BeFalse
+        (Get-Item -LiteralPath $path).LastWriteTimeUtc | Should -Be $before
+    }
+
+    It 'Overwrites when content differs' {
+        $path = Join-Path $script:testDir 'diff-content.txt'
+        Set-Content -Path $path -Value 'old' -Encoding utf8 -NoNewline
+        $result = Set-ContentIfChanged -Path $path -Value 'new'
+        $result | Should -BeTrue
+        Get-Content -Path $path -Raw -Encoding utf8 | Should -Be 'new'
+    }
+
+    It 'Case-sensitive comparison triggers write' {
+        $path = Join-Path $script:testDir 'case-sensitive.txt'
+        Set-Content -Path $path -Value 'Hello' -Encoding utf8 -NoNewline
+        $result = Set-ContentIfChanged -Path $path -Value 'hello'
+        $result | Should -BeTrue
+        Get-Content -Path $path -Raw -Encoding utf8 | Should -Be 'hello'
+    }
+
+    It 'Handles empty string content' {
+        $path = Join-Path $script:testDir 'empty-content.txt'
+        $result = Set-ContentIfChanged -Path $path -Value ''
+        $result | Should -BeTrue
+        [System.IO.File]::ReadAllText($path) | Should -Be ''
+    }
+
+    It 'Writes UTF-8 without BOM and no trailing newline' {
+        $path = Join-Path $script:testDir 'encoding-check.txt'
+        Set-ContentIfChanged -Path $path -Value 'test content' | Out-Null
+        $bytes = [System.IO.File]::ReadAllBytes($path)
+        # UTF-8 BOM is 0xEF 0xBB 0xBF — first bytes must not match
+        if ($bytes.Length -ge 3) {
+            ($bytes[0] -eq 0xEF -and $bytes[1] -eq 0xBB -and $bytes[2] -eq 0xBF) | Should -BeFalse
+        }
+        # No trailing newline
+        $text = [System.IO.File]::ReadAllText($path)
+        $text | Should -Be 'test content'
+    }
+}