Merge pull request #312 from lexming/archive

update documentation about deprecated and archived easyconfigs

Author: branfosj

docs/archived-easyconfigs.md

@@ -3,54 +3,66 @@
 Since EasyBuild v3.0.0, easyconfig files using deprecated (i.e., old and
 inactive) toolchains are *archived*.
 
+Since EasyBuild v5.0.0, archived easyconfig files are transferred to the
+[easybuild-easyconfigs-archive](https://github.com/easybuilders/easybuild-easyconfigs-archive)
+repository.
+
 ## Toolchain deprecation {: #archived_easyconfigs_toolchain_deprecation }
 
-Once in a blue moon, we review the list of toolchains (& versions) that
-are included in EasyBuild.
+Support for toolchains included in EasyBuild is defined in our
+[support policy on toolchain generations][policy_toolchains].
 
-Easyconfig files that use toolchains that become *deprecated* are then
-moved to the *easyconfigs archive*, i.e. the `__archive__` subdirectory
-in the `easybuild-easyconfigs` repository (see
-<https://github.com/easybuilders/easybuild-easyconfigs/tree/main/easybuild/easyconfigs/__archive__>).
+Once a new toolchain generation is released, it will be supported by the
+maintainers of Easybuild for a limited period of time. Afterwards, it becomes
+deprecated and at some point it will be archived including all easyconfig files
+using that toolchain.
 
 ### What are deprecated toolchains? {: #archived_easyconfigs_deprecated_toolchains_what }
 
-Toolchains become deprecated if:
-
-- no easyconfig files using that toolchain have been contributed
-  recently (e.g., in the last year)
-- that toolchain is considered to be inactive, after consulting the
-  EasyBuild community (via mailing list, bi-weekly conf calls)
+A deprecated toolchain is a toolchain version or generation that is no longer
+recommended for use in production systems.
 
-Deprecating a toolchain implies that all easyconfigs using that
-toolchain are moved to the easyconfigs archive, and that they are no
-longer included in the EasyBuild regression test. In addition, these
-easyconfigs are 'hidden' from plain sight, in the sense that you need to
-use `--consider-archived-easyconfigs` to make EasyBuild consider them
-when it is looking for easyconfigs (e.g., with `--search` or
-`--robot`).
+Initially, deprecated toolchains are removed from the EasyBuild regression
+tests and new contributions in the form of easyconfigs become restricted.
+After a period of time defined in our [support policy][policy_toolchains],
+all easyconfig files using that deprecated toolchain are moved to the
+*easyconfigs archive*, i.e. the `__archive__` subdirectory in the
+[easybuild-easyconfigs-archive](https://github.com/easybuilders/easybuild-easyconfigs-archive)
+repository.
 
-This does *not* mean that the support for using these toolchains is
-removed from the EasyBuild framework, although not testing them anymore
+This does *not* mean that the support for using deprecated toolchains is
+removed from the EasyBuild framework, although the lack of testing
 may imply that using them may no longer work at some point in time.
 
-For toolchains for which no active versions are available (outside of
-the easyconfigs archive), it is possible that they will be reactivated,
-if a new toolchain version is contributed.
-
 ### Why are toolchains being deprecated? {: #archived_easyconfigs_deprecated_toolchains_why }
 
-- using old toolchains (incl. old compilers and/or libraries) is
-  likely to become more and more difficult on modern operating systems
-- these toolchains put a significant burden on the regression testing
-  for EasyBuild releases
-- easyconfigs using old toolchains are likely to be for old software
-  versions, which may no longer be relevant anyway
+Toolchain generations are deprecated if:
+
+- it is established in our [support policy][policy_toolchains]
+- they become inactive, meaning that no easyconfig files using that toolchain
+  have been contributed for a long period of time (e.g., in the last year)
+- they become [obsolete][obsolete_easyconfigs_toolchains], which usually
+  happens when a toolchain is replaced by some other newer toolchain
 
-## Using `--consider-archived-easyconfigs` {: #archived_easyconfigs_consider }
+Inactive toolchains may become re-activated in the future if a new toolchain
+version is contributed to `easybuild-easyconfigs`.
 
-To make EasyBuild consider archived easyconfig files, you need to enable
-the `--consider-archived-easyconfigs` configuration option:
+## Using archived easyconfigs {: #use_archived_easyconfigs }
+
+The archive of easyconfigs is not installed by default. You need to install the
+`easybuild-easyconfigs` package with the `archive` extra option in `pip`. This
+will pull in the `easybuild-easyconfigs-archive` package, which provides the
+contents of the `__archive__` subdirectory.
+
+```shell
+python -m pip install easybuild-easyconfigs[archive]
+```
+
+Easyconfig files in the archive are *hidden* from plain sight, meaning that the
+`eb` command will not use those files by default. You need to use the
+`--consider-archived-easyconfigs` configuration option to make EasyBuild look
+into the archive when it seeks easyconfigs (e.g., with `--search` or
+`--robot`).
 
 ``` console
 $ eb -S '^goolfc'

docs/common-toolchains.md

@@ -153,11 +153,16 @@ available in [Overview of common toolchains][common_toolchains_overview].
 
 The intention is to revise and update the common toolchains every 6
 months: once in late December/early January (version `<year>a`), and
-once in late June/early July (version `<year>b`).
+once in late June/early July (version `<year>b`). Each one of these
+versions is a so-called *toolchain generation*.
 
-This is meant be to be a community effort, in the sense that a proposal
-for an updated composition is shared and discussed before it is set in
-stone.
+Each new version is meant be to be a community effort, in the sense that a
+proposal for an updated composition is shared and discussed before it is set in
+stone. Usually, discussion about new toolchains takes place initially in the
+[toolchain-wg](https://easybuild.slack.com/archives/C02FU5Y5CDT) in Slack, the
+bi-weekly conf calls and later in respective pull requests in the
+[easybuild-easyconfigs](https://github.com/easybuilders/easybuild-easyconfigs)
+repository.
 
 Recent versions of each of the toolchain components are considered,
 taking into account stability, performance improvements, added features,
@@ -167,71 +172,69 @@ Moreover, the proposed toolchain compositions are tested extensively,
 typically by rebuilding all available easyconfigs that are using the
 most recent revision of the common toolchains at that time.
 
+Once a new toolchain generation is released, it will be supported by the
+maintainers of Easybuild for a limited period of time. Which is determined by
+our [support policy on toolchain][policy_toolchains].
+
 ## Overview of common toolchains {: #common_toolchains_overview }
 
-### Component versions in `foss` toolchain {: #common_toolchains_overview_foss }
+We support the [last 6 generations][policy_toolchains] of common toolchains.
 
-| `foss`  | *date*   | *binutils* | *GCC*  | *Open MPI* | *FlexiBLAS* | *OpenBLAS* | *LAPACK*               | *ScaLAPACK* | *FFTW* |
-|---------|----------|------------|--------|------------|-------------|------------|------------------------|-------------|--------|
-| `2021a` | May '21  | 2.36.1     | 10.3.0 | 4.1.1      | 3.0.4       | 0.3.15     | (incl. with FlexiBLAS) | 2.1.0       | 3.3.9  |
-| `2021b` | Oct '21  | 2.37       | 11.2.0 | 4.1.1      | 3.0.4       | 0.3.18     | (incl. with FlexiBLAS) | 2.1.0       | 3.3.10 |
-| `2022a` | Jun '22  | 2.38       | 11.3.0 | 4.1.4      | 3.2.0       | 0.3.20     | (incl. with FlexiBLAS) | 2.2.0       | 3.3.10 |
-| `2022b` | Dec '22  | 2.39       | 12.2.0 | 4.1.4      | 3.2.1       | 0.3.21     | (incl. with FlexiBLAS) | 2.2.0       | 3.3.10 |
-| `2023a` | Jun '23  | 2.40       | 12.3.0 | 4.1.5      | 3.3.1       | 0.3.23     | (incl. with FlexiBLAS) | 2.2.0       | 3.3.10 |
-| `2023b` | Dec '23  | 2.40       | 13.2.0 | 4.1.6      | 3.3.1       | 0.3.24     | (incl. with FlexiBLAS) | 2.2.0       | 3.3.10 |
-| `2024a` | Aug '24  | 2.42       | 13.3.0 | 5.0.3      | 3.4.4       | 0.3.27     | (incl. with FlexiBLAS) | 2.2.0       | 3.3.10 |
+Archived toolchains can be found in the
+[easybuild-easyconfigs-archive](https://github.com/easybuilders/easybuild-easyconfigs-archive)
+repository.
 
-### Component versions in `intel` toolchain {: #common_toolchains_overview_intel }
+### Component versions in `foss` toolchain {: #common_toolchains_overview_foss }
 
-| `intel` | *date*   | *binutils* | *GCC*  | *Intel compilers* | *Intel MPI* | *Intel MKL* |
-|---------|----------|------------|--------|-------------------|-------------|-------------|
-| `2021a` | May '21  | 2.36.1     | 10.3.0 | 2021.2.0          | 2021.2.0    | 2021.2.0    |
-| `2021b` | Oct '21  | 2.37       | 11.2.0 | 2021.4.0          | 2021.4.0    | 2021.4.0    |
-| `2022a` | Jun '22  | 2.38       | 11.3.0 | 2022.1.0          | 2021.6.0    | 2022.1.0    |
-| `2022b` | Dec '22  | 2.39       | 12.2.0 | 2022.2.1          | 2021.7.1    | 2022.2.1    |
-| `2023a` | Jun '23  | 2.40       | 12.3.0 | 2023.1.0          | 2021.9.1    | 2023.1.0    |
-| `2023b` | Dec '23  | 2.40       | 13.2.0 | 2023.2.1          | 2021.10.1   | 2023.2.0    |
-| `2024a` | Aug '24  | 2.42       | 13.3.0 | 2024.2.0          | 2021.13.0   | 2024.2.0    |
-
-## Overview of common toolchains (deprecated versions) {: #common_toolchains_overview_deprecated }
-
-### Component versions in `foss` toolchain (deprecated versions) {: #common_toolchains_overview_foss_deprecated }
-
-| `foss`  | *date*   | *binutils* | *GCC*  | *Open MPI* | *FlexiBLAS* | *OpenBLAS* | *LAPACK*               | *ScaLAPACK* | *FFTW*      |
-|---------|----------|------------|--------|------------|-------------|------------|------------------------|-------------|-------------|
-| `2014b` | Jul '14  | '*(none)*  | 4.8.3  | 1.8.1      | *(none)*    | 0.2.9      | 3.5.0                  | 2.0.2       | 3.3.4       |
-| `2015a` | Jan '15  | '*(none)*  | 4.9.2  | 1.8.4      | *(none)*    | 0.2.13     | 3.5.0                  | 2.0.2       | 3.3.4       |
-| `2015b` | Jul '15  | 2.25       | 4.9.3  | 1.8.8      | *(none)*    | 0.2.14     | 3.5.0                  | 2.0.2       | 3.3.4       |
-| `2016a` | Jan '16  | 2.25       | 4.9.3  | 1.10.2     | *(none)*    | 0.2.15     | 3.6.0                  | 2.0.2       | 3.3.4       |
-| `2016b` | Jul '16  | 2.26       | 5.4.0  | 1.10.3     | *(none)*    | 0.2.18     | 3.6.1                  | 2.0.2       | 3.3.4       |
-| `2017a` | Jan '17  | 2.27       | 6.3.0  | 2.0.2      | *(none)*    | 0.2.19     | 3.7.0                  | 2.0.2       | 3.3.6(-pl2) |
-| `2017b` | Jul '17  | 2.28       | 6.4.0  | 2.1.1      | *(none)*    | 0.2.20\*   | (incl. with OpenBLAS)  | 2.0.2       | 3.3.6(-pl2) |
-| `2018a` | Jan '18  | 2.28       | 6.4.0  | 2.1.2      | *(none)*    | 0.2.20\*   | (incl. with OpenBLAS)  | 2.0.2       | 3.3.7       |
-| `2018b` | Jul '18  | 2.30       | 7.3.0  | 3.1.1      | *(none)*    | 0.3.1      | (incl. with OpenBLAS)  | 2.0.2       | 3.3.8       |
-| `2019a` | Jan '19  | 2.31.1     | 8.2.0  | 3.1.3      | *(none)*    | 0.3.5      | (incl. with OpenBLAS)  | 2.0.2       | 3.3.8       |
-| `2019b` | Sept '19 | 2.32       | 8.3.0  | 3.1.4      | *(none)*    | 0.3.7      | (incl. with OpenBLAS)  | 2.0.2       | 3.3.8       |
-| `2020a` | May '20  | 2.34       | 9.3.0  | 4.0.3      | *(none)*    | 0.3.9      | (incl. with OpenBLAS)  | 2.1.0       | 3.3.8       |
-| `2020b` | Nov '20  | 2.35       | 10.2.0 | 4.0.5      | *(none)*    | 0.3.12     | (incl. with OpenBLAS)  | 2.1.0       | 3.3.8       |
+| `foss`  | *support*  | *date*   | *binutils* | *GCC*  | *Open MPI* | *FlexiBLAS* | *OpenBLAS* | *LAPACK*               | *ScaLAPACK* | *FFTW*      |
+|---------|------------|----------|------------|--------|------------|-------------|------------|------------------------|-------------|-------------|
+| `2024a` | active     | Aug '24  | 2.42       | 13.3.0 | 5.0.3      | 3.4.4       | 0.3.27     | (incl. with FlexiBLAS) | 2.2.0       | 3.3.10      |
+| `2023b` | active     | Dec '23  | 2.40       | 13.2.0 | 4.1.6      | 3.3.1       | 0.3.24     | (incl. with FlexiBLAS) | 2.2.0       | 3.3.10      |
+| `2023a` | active     | Jun '23  | 2.40       | 12.3.0 | 4.1.5      | 3.3.1       | 0.3.23     | (incl. with FlexiBLAS) | 2.2.0       | 3.3.10      |
+| `2022b` | active     | Dec '22  | 2.39       | 12.2.0 | 4.1.4      | 3.2.1       | 0.3.21     | (incl. with FlexiBLAS) | 2.2.0       | 3.3.10      |
+| `2022a` | active     | Jun '22  | 2.38       | 11.3.0 | 4.1.4      | 3.2.0       | 0.3.20     | (incl. with FlexiBLAS) | 2.2.0       | 3.3.10      |
+| `2021b` | active     | Oct '21  | 2.37       | 11.2.0 | 4.1.1      | 3.0.4       | 0.3.18     | (incl. with FlexiBLAS) | 2.1.0       | 3.3.10      |
+| `2021a` | deprecated | May '21  | 2.36.1     | 10.3.0 | 4.1.1      | 3.0.4       | 0.3.15     | (incl. with FlexiBLAS) | 2.1.0       | 3.3.9       |
+| `2020b` | deprecated | Nov '20  | 2.35       | 10.2.0 | 4.0.5      | *(none)*    | 0.3.12     | (incl. with OpenBLAS)  | 2.1.0       | 3.3.8       |
+| `2020a` | archived   | May '20  | 2.34       | 9.3.0  | 4.0.3      | *(none)*    | 0.3.9      | (incl. with OpenBLAS)  | 2.1.0       | 3.3.8       |
+| `2019b` | archived   | Sept '19 | 2.32       | 8.3.0  | 3.1.4      | *(none)*    | 0.3.7      | (incl. with OpenBLAS)  | 2.0.2       | 3.3.8       |
+| `2019a` | archived   | Jan '19  | 2.31.1     | 8.2.0  | 3.1.3      | *(none)*    | 0.3.5      | (incl. with OpenBLAS)  | 2.0.2       | 3.3.8       |
+| `2018b` | archived   | Jul '18  | 2.30       | 7.3.0  | 3.1.1      | *(none)*    | 0.3.1      | (incl. with OpenBLAS)  | 2.0.2       | 3.3.8       |
+| `2018a` | archived   | Jan '18  | 2.28       | 6.4.0  | 2.1.2      | *(none)*    | 0.2.20\*   | (incl. with OpenBLAS)  | 2.0.2       | 3.3.7       |
+| `2017b` | archived   | Jul '17  | 2.28       | 6.4.0  | 2.1.1      | *(none)*    | 0.2.20\*   | (incl. with OpenBLAS)  | 2.0.2       | 3.3.6(-pl2) |
+| `2017a` | archived   | Jan '17  | 2.27       | 6.3.0  | 2.0.2      | *(none)*    | 0.2.19     | 3.7.0                  | 2.0.2       | 3.3.6(-pl2) |
+| `2016b` | archived   | Jul '16  | 2.26       | 5.4.0  | 1.10.3     | *(none)*    | 0.2.18     | 3.6.1                  | 2.0.2       | 3.3.4       |
+| `2016a` | archived   | Jan '16  | 2.25       | 4.9.3  | 1.10.2     | *(none)*    | 0.2.15     | 3.6.0                  | 2.0.2       | 3.3.4       |
+| `2015b` | archived   | Jul '15  | 2.25       | 4.9.3  | 1.8.8      | *(none)*    | 0.2.14     | 3.5.0                  | 2.0.2       | 3.3.4       |
+| `2015a` | archived   | Jan '15  | '*(none)*  | 4.9.2  | 1.8.4      | *(none)*    | 0.2.13     | 3.5.0                  | 2.0.2       | 3.3.4       |
+| `2014b` | archived   | Jul '14  | '*(none)*  | 4.8.3  | 1.8.1      | *(none)*    | 0.2.9      | 3.5.0                  | 2.0.2       | 3.3.4       |
 
 *(components marked with* \* *were patched)*
 
-### Component versions in `intel` toolchain (deprecated versions) {: #common_toolchains_overview_intel_deprecated }
-
-| `intel` | *date*   | *binutils* | *GCC*  | *Intel compilers* | *Intel MPI* | *Intel MKL* |
-|---------|----------|------------|--------|-------------------|-------------|-------------|
-| `2014b` | Jul '14  | '*(none)*  | 4.8.3  | 2013.5.192        | 4.1.3.049   | 11.1.2.144  |
-| `2015a` | Jan '15  | '*(none)*  | 4.9.2  | 2015.1.133        | 5.0.2.044   | 11.2.1.133  |
-| `2015b` | Jul '15  | 2.25       | 4.9.3  | 2015.3.187        | 5.0.3.048   | 11.2.3.187  |
-| `2016a` | Jan '16  | 2.26       | 4.9.3  | 2016.1.150        | 5.1.2.150   | 11.3.1.150  |
-| `2016b` | Jul '16  | 2.26       | 5.4.0  | 2016.3.210        | 5.1.3.181   | 11.3.3.210  |
-| `2017a` | Jan '17  | 2.27       | 6.3.0  | 2017.1.132        | 2017.1.132  | 2017.1.132  |
-| `2017b` | Jul '17  | 2.28       | 6.4.0  | 2017.4.196        | 2017.3.196  | 2017.3.196  |
-| `2018a` | Jan '18  | 2.28       | 6.4.0  | 2018.1.163        | 2018.1.163  | 2018.1.163  |
-| `2018b` | Jul '18  | 2.30       | 7.3.0  | 2018.3.222        | 2018.3.222  | 2018.3.222  |
-| `2019a` | Jan '19  | 2.31.1     | 8.2.0  | 2019.1.144        | 2018.4.274  | 2019.1.144  |
-| `2019b` | Sept '19 | 2.32       | 8.3.0  | 2019.5.281        | 2018.5.288  | 2019.5.281  |
-| `2020a` | May '20  | 2.34       | 9.3.0  | 2020.1.217        | 2019.7.217  | 2020.1.217  |
-| `2020b` | Nov '20  | 2.35       | 10.2.0 | 2020.4.304        | 2019.9.304  | 2020.4.304  |
+### Component versions in `intel` toolchain {: #common_toolchains_overview_intel }
+
+| `intel` | *support*  | *date*   | *binutils* | *GCC*  | *Intel compilers* | *Intel MPI* | *Intel MKL* |
+|---------|------------|----------|------------|--------|-------------------|-------------|-------------|
+| `2024a` | active     | Aug '24  | 2.42       | 13.3.0 | 2024.2.0          | 2021.13.0   | 2024.2.0    |
+| `2023b` | active     | Dec '23  | 2.40       | 13.2.0 | 2023.2.1          | 2021.10.1   | 2023.2.0    |
+| `2023a` | active     | Jun '23  | 2.40       | 12.3.0 | 2023.1.0          | 2021.9.1    | 2023.1.0    |
+| `2022b` | active     | Dec '22  | 2.39       | 12.2.0 | 2022.2.1          | 2021.7.1    | 2022.2.1    |
+| `2022a` | active     | Jun '22  | 2.38       | 11.3.0 | 2022.1.0          | 2021.6.0    | 2022.1.0    |
+| `2021b` | active     | Oct '21  | 2.37       | 11.2.0 | 2021.4.0          | 2021.4.0    | 2021.4.0    |
+| `2021a` | deprecated | May '21  | 2.36.1     | 10.3.0 | 2021.2.0          | 2021.2.0    | 2021.2.0    |
+| `2020b` | deprecated | Nov '20  | 2.35       | 10.2.0 | 2020.4.304        | 2019.9.304  | 2020.4.304  |
+| `2020a` | archived   | May '20  | 2.34       | 9.3.0  | 2020.1.217        | 2019.7.217  | 2020.1.217  |
+| `2019b` | archived   | Sept '19 | 2.32       | 8.3.0  | 2019.5.281        | 2018.5.288  | 2019.5.281  |
+| `2019a` | archived   | Jan '19  | 2.31.1     | 8.2.0  | 2019.1.144        | 2018.4.274  | 2019.1.144  |
+| `2018b` | archived   | Jul '18  | 2.30       | 7.3.0  | 2018.3.222        | 2018.3.222  | 2018.3.222  |
+| `2018a` | archived   | Jan '18  | 2.28       | 6.4.0  | 2018.1.163        | 2018.1.163  | 2018.1.163  |
+| `2017b` | archived   | Jul '17  | 2.28       | 6.4.0  | 2017.4.196        | 2017.3.196  | 2017.3.196  |
+| `2017a` | archived   | Jan '17  | 2.27       | 6.3.0  | 2017.1.132        | 2017.1.132  | 2017.1.132  |
+| `2016b` | archived   | Jul '16  | 2.26       | 5.4.0  | 2016.3.210        | 5.1.3.181   | 11.3.3.210  |
+| `2016a` | archived   | Jan '16  | 2.26       | 4.9.3  | 2016.1.150        | 5.1.2.150   | 11.3.1.150  |
+| `2015b` | archived   | Jul '15  | 2.25       | 4.9.3  | 2015.3.187        | 5.0.3.048   | 11.2.3.187  |
+| `2015a` | archived   | Jan '15  | '*(none)*  | 4.9.2  | 2015.1.133        | 5.0.2.044   | 11.2.1.133  |
+| `2014b` | archived   | Jul '14  | '*(none)*  | 4.8.3  | 2013.5.192        | 4.1.3.049   | 11.1.2.144  |
 
 ## Customizing common toolchains {: #common_toolchains_customizing }