[main] Internal CLR error (0x80131506) when building Fedora41_Offline_CurrentSourceBuiltSdk_x64

[ellahathaway]

Encountered internal CLR errors on Fedora41_Offline_CurrentSourceBuiltSdk_x64 leg during runtime repo build.

Link to failure

Error Output

/__w/1/s/src/runtime/artifacts/bin/Crossgen2Tasks/Release/net10.0/Microsoft.NET.CrossGen.targets(464,5): error : Fatal error. [/__w/1/s/src/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.CoreCLR.sfxproj]
/__w/1/s/src/runtime/artifacts/bin/Crossgen2Tasks/Release/net10.0/Microsoft.NET.CrossGen.targets(464,5): error : Internal CLR error. (0x80131506) [/__w/1/s/src/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.CoreCLR.sfxproj]
... (see full error log above)
/__w/1/s/src/runtime/artifacts/bin/Crossgen2Tasks/Release/net10.0/Microsoft.NET.CrossGen.targets(352,5): error NETSDK1096: Optimizing assemblies for performance failed. You can either exclude the failing assemblies from being optimized, or set the PublishReadyToRun property to false. [/__w/1/s/src/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.CoreCLR.sfxproj]

The error is repeatable.

[ellahathaway]

@dotnet/crossgen-contrib - could you please take a look? Thanks in advance.

[mangod9]

do we know what change might have triggered the failure? Also is there a specific container image which can use to repro?

[ellahathaway]

do we know what change might have triggered the failure?

It's unclear. The failure first appeared internally with dotnet/dotnet@1c92f34, but the public validation passed.

Also is there a specific container image which can use to repro?

Yes: mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-41

[mangod9]

@steveisok was trying to get a dump, but looks like the issue is intermittent ?

[ellahathaway]

looks like the issue is intermittent ?

This morning's build is the first internal build I've seen without the failure. Before that, we were hitting the error on every internal build since Saturday (5 builds in a row).

[mthalman]

It is intermittent. In this build (internal link), it originally failed but passed on a retry.

[jkotas]

@ellahathaway @mthalman We need crash dump to make progress on diagnosing this (set DOTNET_DbgEnableMiniDump=1 env variable - https://learn.microsoft.com/en-us/dotnet/core/diagnostics/collect-dumps-crash). Also, make sure to save the binaries and symbols that match the crash dump.

[MichaelSimons]

@ellahathaway @mthalman We need crash dump to make progress on diagnosing this (set DOTNET_DbgEnableMiniDump=1 env variable - https://learn.microsoft.com/en-us/dotnet/core/diagnostics/collect-dumps-crash). Also, make sure to save the binaries and symbols that match the crash dump.

IMO, We should have an easy way to enable crash dumps in our CI - they don't need to be enabled by default, rather something that we can queue a run and set a parameter.