[ATFE] Package common multilib headers into target triple directory. #423
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dcandler
reviewed
Jul 16, 2025
dcandler
reviewed
Jul 23, 2025
The current ATFE package redundantly duplicates identical C and C++ headers across more than 100 ibrary variants, despite the fact that these headers do not vary within a given target triple aside from few. As a result, each variant currently maintains its own include/ and include/c++/v1/ directories, leading to unnecessary duplication. This patch is to refactor the multilib layout to use a single shared header directory per target triple, eliminating redundant copies of identical headers across library variants, while keeping variant-specific headers within each variant’s own include/ directory. Changing the layout of the package structure will require corresponding updates to the ATFE build system, as it currently assumes a variant specific directory hierarchy for headers and libraries during installation and packaging. A new Python script has been introduced that runs after the runtime subproject has been executed for each variant and the non-optimised multilib directories have been generated. This script identifies and extracts common headers from the non-optimised multilibs and creates an optimised multilib directory. In this optimised directory, only the common headers are centralised, while the remaining contents are preserved as-is from the non-optimised layout. A CMake flag has been added to disable this multilib optimisation phase. When this flag is set, the optimised multilib directory will not be generated. Additionally, the optimisation step is skipped automatically if there are fewer than two multilib variants to build. To support this new layout with a centralised include directory, a new field called IncludePath has been added to multilib.yaml.This field specifies a base path for locating header files wich gets added to the header search path sequence by clang. Corresponding changes in Clang: llvm/llvm-project#146651
…er than the variant comparison step.
…e file. The script that performs the common header generation assumes the multilib YAML file is already generated. This change adds an explicit dependency on the multilib-yaml target to ensure the file exists before the common header generate step is triggered.
Only enable IncludePaths in multilib.yaml if header optimization is turned on.
dcandler
reviewed
Jul 23, 2025
At present ENABLE_MULTILIB_HEADER_OPTIMISATION has to be disabled as this feature is not fully supported from the toolchain yet.
dcandler
reviewed
Jul 25, 2025
dcandler
previously approved these changes
Jul 28, 2025
dcandler
previously approved these changes
Jul 28, 2025
dcandler
approved these changes
Jul 28, 2025
simpal01
added a commit
to simpal01/arm-toolchain
that referenced
this pull request
Jul 31, 2025
…n IncludeDirs field. (arm#447) Downstream issue:arm#446. This patch extends the Multilib yaml format to allow specifying an IncludeDirs for the header path/s per multilib variant. The goal is to enable fine-grained control over header search paths for each multilib variant. This feature is especially useful in setups where header paths deviate from the default bare-metal assumptions. For example, when headers are shared across target triples, it becomes necessary to organise them under target-triple-specific directories to ensure correct resolution. This is already in upstream review llvm/llvm-project#146651. Since the review is pending, a downstream patch is made to enable IncludeDirs support in multilib.yaml [ATFE] Package common multilib headers into target triple directory. (arm#423) The current ATFE package redundantly duplicates identical C and C++ headers across more than 100 ibrary variants, despite the fact that these headers do not vary within a given target triple aside from few. As a result, each variant currently maintains its own include/ and include/c++/v1/ directories, leading to unnecessary duplication. This patch is to refactor the multilib layout to use a single shared header directory per target triple, eliminating redundant copies of identical headers across library variants, while keeping variant-specific headers within each variant’s own include/ directory. Changing the layout of the package structure will require corresponding updates to the ATFE build system, as it currently assumes a variant specific directory hierarchy for headers and libraries during installation and packaging. A new Python script has been introduced that runs after the runtime subproject has been executed for each variant and the non-optimised multilib directories have been generated. This script identifies and extracts common headers from the non-optimised multilibs and creates an optimized multilib directory. In this optimised directory, only the common headers are centralised, while the remaining contents are preserved as-is from the non-optimised layout. A CMake flag called ENABLE_MULTILIB_HEADER_OPTIMISATION has been added to disable this multilib optimisation phase. When this flag is not set, the optimised multilib directory will not be generated. Additionally, the optimisation step is skipped automatically if there are fewer than two multilib variants to build. To support this new layout with a centralised include directory, a new field called IncludeDirs has been added to multilib.yaml. This field specifies a base directory for locating header files which gets added to the header search path sequence by clang. Corresponding changes in Clang: llvm/llvm-project#146651 [ATFE] Prioritize multilib-specific headers over target-level headers. (arm#451) Adjust the header search order so that variant-specific directories (e.g armv6m_soft_nofp_size/include/) are searched before the common target-level directories (e.g include/) in the -internal-isystem list. This ensures that headers tailored for a specific multilib variant take precedence over the generic headers, allowing correct header resolution in the presence of overrides. Previously, the common target-level directory could shadow multilib-specific headers if the same file existed in both paths.
simpal01
added a commit
to simpal01/arm-toolchain
that referenced
this pull request
Jul 31, 2025
…rm#423) The current ATFE package redundantly duplicates identical C and C++ headers across more than 100 ibrary variants, despite the fact that these headers do not vary within a given target triple aside from few. As a result, each variant currently maintains its own include/ and include/c++/v1/ directories, leading to unnecessary duplication. This patch is to refactor the multilib layout to use a single shared header directory per target triple, eliminating redundant copies of identical headers across library variants, while keeping variant-specific headers within each variant’s own include/ directory. Changing the layout of the package structure will require corresponding updates to the ATFE build system, as it currently assumes a variant specific directory hierarchy for headers and libraries during installation and packaging. A new Python script has been introduced that runs after the runtime subproject has been executed for each variant and the non-optimised multilib directories have been generated. This script identifies and extracts common headers from the non-optimised multilibs and creates an optimized multilib directory. In this optimised directory, only the common headers are centralised, while the remaining contents are preserved as-is from the non-optimised layout. A CMake flag called ENABLE_MULTILIB_HEADER_OPTIMISATION has been added to disable this multilib optimisation phase. When this flag is not set, the optimised multilib directory will not be generated. Additionally, the optimisation step is skipped automatically if there are fewer than two multilib variants to build. To support this new layout with a centralised include directory, a new field called IncludeDirs has been added to multilib.yaml. This field specifies a base directory for locating header files which gets added to the header search path sequence by clang. Corresponding changes in Clang: llvm/llvm-project#146651
simpal01
added a commit
to simpal01/arm-toolchain
that referenced
this pull request
Jul 31, 2025
…rm#423) The current ATFE package redundantly duplicates identical C and C++ headers across more than 100 ibrary variants, despite the fact that these headers do not vary within a given target triple aside from few. As a result, each variant currently maintains its own include/ and include/c++/v1/ directories, leading to unnecessary duplication. This patch is to refactor the multilib layout to use a single shared header directory per target triple, eliminating redundant copies of identical headers across library variants, while keeping variant-specific headers within each variant’s own include/ directory. Changing the layout of the package structure will require corresponding updates to the ATFE build system, as it currently assumes a variant specific directory hierarchy for headers and libraries during installation and packaging. A new Python script has been introduced that runs after the runtime subproject has been executed for each variant and the non-optimised multilib directories have been generated. This script identifies and extracts common headers from the non-optimised multilibs and creates an optimized multilib directory. In this optimised directory, only the common headers are centralised, while the remaining contents are preserved as-is from the non-optimised layout. A CMake flag called ENABLE_MULTILIB_HEADER_OPTIMISATION has been added to disable this multilib optimisation phase. When this flag is not set, the optimised multilib directory will not be generated. Additionally, the optimisation step is skipped automatically if there are fewer than two multilib variants to build. To support this new layout with a centralised include directory, a new field called IncludeDirs has been added to multilib.yaml. This field specifies a base directory for locating header files which gets added to the header search path sequence by clang. Corresponding changes in Clang: llvm/llvm-project#146651
simpal01
added a commit
that referenced
this pull request
Aug 1, 2025
…423) The current ATFE package redundantly duplicates identical C and C++ headers across more than 100 ibrary variants, despite the fact that these headers do not vary within a given target triple aside from few. As a result, each variant currently maintains its own include/ and include/c++/v1/ directories, leading to unnecessary duplication. This patch is to refactor the multilib layout to use a single shared header directory per target triple, eliminating redundant copies of identical headers across library variants, while keeping variant-specific headers within each variant’s own include/ directory. Changing the layout of the package structure will require corresponding updates to the ATFE build system, as it currently assumes a variant specific directory hierarchy for headers and libraries during installation and packaging. A new Python script has been introduced that runs after the runtime subproject has been executed for each variant and the non-optimised multilib directories have been generated. This script identifies and extracts common headers from the non-optimised multilibs and creates an optimized multilib directory. In this optimised directory, only the common headers are centralised, while the remaining contents are preserved as-is from the non-optimised layout. A CMake flag called ENABLE_MULTILIB_HEADER_OPTIMISATION has been added to disable this multilib optimisation phase. When this flag is not set, the optimised multilib directory will not be generated. Additionally, the optimisation step is skipped automatically if there are fewer than two multilib variants to build. To support this new layout with a centralised include directory, a new field called IncludeDirs has been added to multilib.yaml. This field specifies a base directory for locating header files which gets added to the header search path sequence by clang. Corresponding changes in Clang: llvm/llvm-project#146651
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The current ATFE package redundantly duplicates identical C and C++ headers across more than 100 ibrary variants, despite the fact that these headers do not vary within a given target triple aside from few. As a result, each variant currently maintains its own include/ and include/c++/v1/ directories, leading to unnecessary duplication.
This patch is to refactor the multilib layout to use a single shared header directory per target triple, eliminating redundant copies of identical headers across library variants, while keeping variant-specific headers within each variant’s own include/ directory.
Changing the layout of the package structure will require corresponding updates to the ATFE build system, as it currently assumes a variant specific directory hierarchy for headers and libraries during installation and packaging.
A new Python script has been introduced that runs after the runtime subproject has been executed for each variant and the non-optimised multilib directories have been generated. This script identifies and extracts common headers from the non-optimised multilibs and creates an optimized multilib directory. In this optimised directory, only the common headers are centralised, while the remaining contents are preserved as-is from the non-optimised layout.
A CMake flag called ENABLE_MULTILIB_HEADER_OPTIMISATION has been added to disable this multilib optimisation phase. When this flag is not set, the optimised multilib directory will not be generated. Additionally, the optimisation step is skipped automatically if there are fewer than two multilib variants to build.
To support this new layout with a centralised include directory, a new field called IncludeDirs has been added to multilib.yaml. This field specifies a base directory for locating header files which gets added to the header search path sequence by clang.
Corresponding changes in Clang:
llvm/llvm-project#146651