[ATFE] Centralising common headers across multilib variants.

[simpal01]

This is a cherry-pick from the main branch that combines three commits.

[Downstream change][Multilib] Extend the Multilib system to support an IncludeDirs field. (#447)
[ATFE] Package common multilib headers into target triple directory. (#423)
[ATFE] Prioritize multilib-specific headers over target-level headers. (#451)

The current ATFE package redundantly duplicates identical C and C++ headers across more than 100 library variants, despite the fact that these headers do not vary within a given target triple aside from few. As a result, each variant currently maintains its own include/ and include/c++/v1/ directories, leading to unnecessary duplication.

This patch is to refactor the multilib layout to use a single shared header directory per target triple, eliminating redundant copies of identical headers across library variants, while keeping variant-specific headers within each variant’s own include/ directory.

Changing the layout of the package structure will require corresponding updates to the ATFE build system, as it currently assumes a variant specific directory hierarchy for headers and libraries during installation and packaging.

This patch extends the Multilib yaml format to allow specifying an IncludeDirs for the header path/s per multilib variant. The goal is to enable fine-grained control over header search paths for each multilib variant. This feature is especially useful in setups where header paths deviate from the default bare-metal assumptions. For example, when headers are shared across target triples, it becomes necessary to organise them under target-triple-specific directories to ensure correct resolution.

This is already in upstream review
llvm/llvm-project#146651.

Since the review is pending, a downstream patch is made to enable IncludeDirs support in multilib.yaml

A new Python script has been introduced in the ATFE build system that runs after the runtime subproject has been executed for each variant and the non-optimised multilib directories have been generated. This script identifies and extracts common headers from the non-optimised multilibs and creates an optimized multilib directory. In this optimised directory, only the common headers are centralised, while the remaining contents are preserved as-is from the non-optimised layout.

A CMake flag called ENABLE_MULTILIB_HEADER_OPTIMISATION has been added to disable this multilib optimisation phase. When this flag is not set, the optimised multilib directory will not be generated. Additionally, the optimisation step is skipped automatically if there are fewer than two multilib variants to build.

To support this new layout with a centralised include directory, a new field called IncludeDirs has been added to multilib.yaml. This field specifies a base directory for locating header files which gets added to the header search path sequence by clang.

Corresponding changes in Clang:
llvm/llvm-project#146651