Skip to content

Conversation

jkoritzinsky
Copy link
Member

Today, crossgen2 uses its own PE builder built on top of System.Reflection.PortableExecutable. However, this implementation ends up manually redoing the vast majority of the features System.Reflection.PortableExecutable provides manually to support a number of edge cases that the library does not provide. Additionally, we plan to introduce a ReadyToRun format based on Mach-O (see #120065). There is no library like System.Reflection.PortableExecutable with a similar API that we could use for the Mach-O format, so we will either need to add some parallel path or move the PE path to something that can be shared with the Mach-O support.

For NativeAOT's ILCompiler, we use an ObjectWriter library based loosely around LibObjectFile to enable us to write COFF, ELF, and Mach-O objects. This library provides a consistent API for all three formats.

This PR changes crossgen2 to use a PE writer based on the COFF object writer from our ObjectWriter library. Additionally, this PR adds a number of features to the ObjectWriter library to support various constructs that are necessary for emitting a ReadyToRun image (symbol ranges, node size "relocs", deterministic file checksums, etc.).

This should make completing #120065 much easier as the vast majority of the "How do we emit code/data correctly so we can meet Apple's requirements?" question is already met by the MachObjectWriter in the ObjectWriter library, leaving a much smaller tail of work when adding support for it.

… need real unwind/debug info for R2R in the same way that NAOT does
…we needed this are CodeView debug sections, which aren't supported in PE files
…to ObjectWriter and move R2R usage over to be based on that.
…of it) can be used from within the ObjectWriter when provided.
…te a file on disk in the new path. (Does it work? That's the next step)
@jkoritzinsky
Copy link
Member Author

/azp list

Copy link

CI/CD Pipelines for this repository:

@jkoritzinsky
Copy link
Member Author

/azp run runtime-coreclr r2r-extra, runtime-coreclr r2r, runtime-coreclr crossgen2, runtime-coreclr crossgen2 outerloop, runtime-coreclr crossgen2-composite, runtime-nativeaot-outerloop

Copy link

Azure Pipelines successfully started running 6 pipeline(s).

@jkoritzinsky jkoritzinsky marked this pull request as ready for review October 13, 2025 20:30
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR converts crossgen2 to use an ObjectWriter-based PE writer for ReadyToRun image generation, replacing the previous PE builder implementation. This change aims to unify the code generation infrastructure and prepare for supporting Mach-O format ReadyToRun images.

Key Changes

  • Replaces the existing PE builder with a new PEObjectWriter that reuses COFF object writer infrastructure
  • Adds support for ReadyToRun container format selection via command line option
  • Removes several large files (SectionBuilder.cs, R2RPEBuilder.cs, RelocationHelper.cs) containing custom PE building logic
  • Updates various ReadyToRun nodes to use proper section assignments and new ObjectWriter APIs

Reviewed Changes

Copilot reviewed 76 out of 84 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/coreclr/tools/aot/crossgen2/Program.cs Adds UseContainerFormat call for ReadyToRun compilation
src/coreclr/tools/aot/crossgen2/Crossgen2RootCommand.cs Adds OutputFormat command line option with PE format support
src/coreclr/tools/aot/ILCompiler.ReadyToRun/ObjectWriter/SectionBuilder.cs Removed - large PE building implementation
src/coreclr/tools/aot/ILCompiler.ReadyToRun/ObjectWriter/R2RPEBuilder.cs Removed - PE builder class
src/coreclr/tools/aot/ILCompiler.ReadyToRun/ObjectWriter/RelocationHelper.cs Removed - relocation helper class
src/coreclr/tools/aot/ILCompiler.ReadyToRun/CodeGen/ReadyToRunObjectWriter.cs Major refactoring to use PEObjectWriter instead of custom PE builder
Various ReadyToRun node files Updates to use proper section assignments and new ObjectWriter APIs
src/coreclr/tools/Common/Compiler/ObjectWriter/PEObjectWriter.cs New - PE image writer built on COFF infrastructure

@jkoritzinsky
Copy link
Member Author

/azp run runtime-coreclr r2r-extra, runtime-coreclr r2r, runtime-coreclr crossgen2, runtime-coreclr crossgen2 outerloop, runtime-coreclr crossgen2-composite

Copy link

Azure Pipelines successfully started running 5 pipeline(s).

@jkoritzinsky
Copy link
Member Author

/azp run runtime-coreclr r2r-extra, runtime-coreclr r2r, runtime-coreclr crossgen2

Copy link

Azure Pipelines successfully started running 3 pipeline(s).

@jkoritzinsky
Copy link
Member Author

/azp run runtime-coreclr r2r-extra, runtime-coreclr r2r, runtime-coreclr crossgen2, runtime-coreclr crossgen2 outerloop, runtime-coreclr crossgen2-composite

Copy link

Azure Pipelines successfully started running 5 pipeline(s).

@jkoritzinsky
Copy link
Member Author

My analysis of the failures:

crossgen2-composite is in main: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1173552&view=ms.vss-test-web.build-test-results-tab

r2r-extra: There's inconsistent occurrences of flaky tests like the ones I'm seeing here (stackoverflowtester in particular shows up quite a bit).

The TestReadyToRun_Libraries runs were broken due to an infra issue. Running them again now.

@jkoritzinsky
Copy link
Member Author

/azp run runtime-coreclr crossgen2

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@jkoritzinsky jkoritzinsky added this to the 11.0.0 milestone Oct 14, 2025
@jkoritzinsky jkoritzinsky moved this to High Priority in AppModel Oct 14, 2025
@jkoritzinsky jkoritzinsky requested a review from janvorli October 15, 2025 23:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: High Priority

Development

Successfully merging this pull request may close these issues.

2 participants