Improve source indexer deduplication to prefer real implementation over *.notsupported.cs files

[Copilot]

This PR addresses the issue where https://source.dot.net/ shows platform-agnostic source code containing mostly empty classes that throw PlatformNotSupportedException instead of the actual platform-specific implementation files.

Problem

The current deduplication logic in BinLogToSln/Program.cs was too simplistic, using only file path-based deduplication. When multiple builds existed for the same assembly (e.g., generic + Linux-specific), it would select whichever was processed first, often resulting in .notsupported.cs stub files being chosen over real implementation files.

Solution

Implemented improved deduplication logic that:

1. Groups by Assembly Name: All compiler invocations are now grouped by AssemblyName instead of simple file path deduplication
2. Scores Each Build: Uses a 5-tier scoring system to select the "best" build based on:
   • UseForSourceIndex property: int.MaxValue score for builds explicitly marked for indexing
   • Platform support status: -10,000 penalty for IsPlatformNotSupportedAssembly builds
   • Framework version: Higher scores for newer frameworks (major1000 + minor100)
   • Platform specificity: +500 bonus for platform-specific frameworks, with additional bonuses for Linux (+100) and Unix (+50)
   • Source file count: +1 per source file as a tiebreaker
3. Selects Best Build: For each assembly, only the highest-scored build is included in the solution

Documentation and Testing

• Documentation: Added comprehensive documentation in docs/source-selection-algorithm.md explaining the scoring priorities, examples with concrete scores, and configuration options
• Unit Tests: Added BinLogToSln.Tests project with 12 test methods covering all scoring scenarios, edge cases, and priority ordering to protect against regressions

Example

Before this change, if there were two builds for System.Net.NameResolution:

• Generic build: System.Net.NameResolution.notsupported.cs (score: -1,955)
• Linux build: actual implementation files (score: 8,727)

The tool would pick whichever was processed first. Now it consistently selects the Linux build with real implementation.

The changes maintain backward compatibility while significantly improving source quality on https://source.dot.net/.

Fixes #188.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[ericstj]

@copilot please try plumbing project properties from the binlog as suggested in the feedback.

[Copilot]

@copilot please try plumbing project properties from the binlog as suggested in the feedback.

Implemented project properties plumbing as requested. The BinLogReader now captures project properties from the binlog and stores them in a ProjectProperties dictionary on CompilerInvocation. The scoring logic uses these properties instead of file/command-line conventions. Commit c59f2a1

[ericstj]

@copilot give this a try.

[Copilot]

@copilot give this a try.

Implemented all the requested changes! The BinLogReader now optimizes project access, the scoring system follows the proper priority hierarchy, and platform-specific precedence has been removed. All builds are passing. Commit 56384ee

[Copilot]

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

• https://api.github.com/repos/KirillOsenkov/MSBuildStructuredLog/commits/00926a0debc2facd65611f68f91adafe0d56ab14
  • Triggering command: curl -s REDACTED (http block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[ericstj]

@joperezr @radical this is ready for review.

This tool actually runs inside the repo builds, for example https://dev.azure.com/dnceng/internal/_build/results?buildId=2771328&view=logs&j=b03b9442-9455-54c1-ae98-43ca74c453ac&t=9dac64f1-1058-5d29-d0d5-2337e1b09e97&l=14
So it's not possible to test with a manual build of the index.

I did test this by running the tool against a dotnet/runtime binlog and examining the SLN it generated. It looked good to me, selecting the right project / source files for the index.

[joperezr]

LGTM barring those two last comments I left. Thanks @ericstj

[ericstj]

@copilot can you address @joperezr's feedback?

[Copilot]

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

• dnceng.pkgs.visualstudio.com
  • Triggering command: dotnet build src/SourceBrowser/src/BinLogToSln/BinLogToSln.csproj (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[Copilot]

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

• dnceng.pkgs.visualstudio.com
  • Triggering command: dotnet test src/SourceBrowser/src/BinLogToSln.Tests/BinLogToSln.Tests.csproj --logger console;verbosity=normal (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[Copilot]

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

• dnceng.pkgs.visualstudio.com
  • Triggering command: dotnet build src/SourceBrowser/src/BinLogParser/BinLogParser.csproj (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[joperezr]

Great, this looks ready to go! Thanks @ericstj!

[ericstj]

Almost, we need to enable tests to run. We should add test step to https://github.com/dotnet/source-indexer/blob/main/.github/workflows/dotnet.yml