Support SoC and board extensions #72857

tejlmand · 2024-05-16T08:59:10Z

Fixes: #69548
Fixes: #72374

This PR introduces support for extending boards and SoC with new variants oot.

Scope reduction in this PR compared to #69548:

Thus it must be possible to describe when a variant want to apply base variant overlay and conf files.
This feature part will be addressed as follow-up PR.

nashif · 2024-05-22T13:03:13Z

nice. would be great if this feature is presented in the arch wg to get more feedback and input.

henrikbrixandersen · 2024-05-28T16:00:08Z

@PerMac Couldn't this be used for addressing #73421 as well?

PerMac · 2024-05-29T10:27:59Z

@PerMac Couldn't this be used for addressing #73421 as well?

I will sync with @tejlmand to get a better understanding.

57300

Tested this a bunch over the past few weeks and it works as expected. Thanks!
Just a couple of comments on the implementation.

cmake/modules/dts.cmake

cmake/modules/kconfig.cmake

57300 · 2024-06-05T17:20:15Z

doc/_extensions/zephyr/kconfig/__init__.py

If board/SoC extensions will be allowed in-tree, will this have to be updated to include all folders?

Suggested change

soc_folders = {soc.folder[0] for soc in v2_systems.get_socs()}

soc_folders = {folder for soc in v2_systems.get_socs() for folder in soc.folder}

Same with the occurrences of board.dir[0].

that depends on how we want to organize the docs in such case.

Will we want docs describing the in-tree soc / board to be located in a single folder, or do we want to allow documentation for a given soc / board to be placed at multiple locations.

I would like to leave such decision to doc maintainers and not as part of this PR.
@kartben thoughts ?

OK, I would agree that the docs can be handled later, same with compliance.

scripts/ci/check_compliance.py

scripts/list_boards.py

57300 · 2024-06-05T17:23:54Z

tests/cmake/hwm/board_extend/oot_root/boards/arm/mps2/mps2_an521-common.dtsi

Wouldn't it be better to include the original board DTS, instead of copying it into the test?

a copy was chosen as we just need a device tree where we can verify that it's being correctly extended.
We have no intention of tracking the real board, and only reason to have the real board in this case is to ensure the extension mechanism works.

We have no intention of tracking the real board

If that's the case, have you considered using something closer to blank/dummy files? I don't know if the dts contents here are essential to pass these tests.

I did, but this is where I took the short route of a copy and instead focus on the board extend functionality itself.

Having the dts as close to the board it extends is probably more likely than an blank / dummy dts file.
Although you can say that for a real extend, a user would most likely try to include a dtsi file if possible, which is close to your first comment 😉

tejlmand · 2024-10-25T13:16:19Z

why is this in a different PR? This should come as part of the same PR and go in together.

doc commit cherry-picked into this PR.

carlescufi · 2024-10-25T13:39:04Z

@kartben regardless of when this gets merged, are you happy with the doc?

kartben

@kartben regardless of when this gets merged, are you happy with the doc?

@carles it looks ok - one repeated paragraph tho

kartben · 2024-10-25T15:45:38Z

doc/hardware/porting/board_porting.rst

repeated. I tend to like second paragraph's formulation better, fwiw
also would recommend skipping a line so that "Extending an existing board with [...]" is another paragraph

of course you like the second paragraph best, cause you wrote it yourself 😉
I happened to just apply your suggestion in this comment #73409 (comment) using the commit suggestion feature of github.

Guess I should have noticed that you were actually not removing the old lines in that suggestion 😆

Repeated paragraph removed.

Extend the board porting guide with documentation on the new board extend feature. Signed-off-by: Torsten Rasmussen <[email protected]>

henrikbrixandersen

I often use the existing board extension mechanism, but it is very limited. This will solve my current issues with that, and open up for much smoother (customised) hardware-in-the-loop testing here. Thanks.

kartben

+1 for docs

nashif · 2024-10-26T11:30:59Z

doc/hardware/porting/board_porting.rst

+   ├── Kconfig
+   ├── Kconfig.plank
+   ├── Kconfig.defconfig
+   └── plank_<new-qualifiers>.yaml


had to scratch my head a bit here trying to understand what the new_qualifier would end up like by looking at the tests. It will help alot if you provide an example here showing how you get to the new qualifier by extending a board.

maybe:

As an example, when extending the native_sim board using the following:

board: extend: native_sim variants: - name: one qualifier: native - name: two qualifier: native/64

the new board targets will end up as native_sim/native/one and native_sim/native/64/two. The DTS files for the extended boards will be native_sim_native_64_two.dts and native_sim_native_one.dts.

nashif · 2024-10-26T11:31:36Z

doc/hardware/porting/board_porting.rst

 Board extensions
 ****************

+The board hardware model in Zephyr allows you to extend an existing board with


The PR also does add SoC extension out of tree, where is this documented?

nashif

please submit followup PRs expanding the docs:

add an example with real names o illustrate how a new board extension will look like when it comes to names
add docs about SoC extensions

Upstream PR: zephyrproject-rtos/zephyr#72857 Cleanup the Kconfig generating code in hwm_v2.cmake by moving common logic inside the kconfig_gen() helper function. This prepares the code for board extension feature. Signed-off-by: Torsten Rasmussen <[email protected]> (cherry picked from commit d39cc46b56294b6eb297a67478334c23db8c8ca3) (cherry picked from commit a20616f) (cherry picked from commit 80f0145)

Upstream PR: zephyrproject-rtos/zephyr#72857 Fixes: #69548 Support extending an existing board with new board variants. This commit introduces the following changes to allow a board to be extended out-of-tree. The board yaml schema is extended to support an extend field which will be used to identify the board to be extended. A board 'plank' can be extended like this: > board: > extend: plank > variants: > - name: ext > qualifier: soc1 For the rest of the build system this means that there is no longer a single board directory. The existing CMake variable BOARD_DIR is kept and reference the directory which defines the board. A new CMake variable BOARD_DIRECTORIES provides a list of all directories which defines board targets for the board. This means the directory which defines the board as well as all directories that extends the board. Signed-off-by: Torsten Rasmussen <[email protected]> (cherry picked from commit ad16407aca8ccde1aaa57d4d1dcb15bb4d0bfc25) (cherry picked from commit a276e94) (cherry picked from commit ad3a56e)

Upstream PR: zephyrproject-rtos/zephyr#72857 Fixes: #72374 Support extending an existing SoC with new CPU clusters. This commit introduces the following changes to allow an SoC to be extended out-of-tree. The SoC yaml schema is extended to support an extend field which will be used to identify the SoC to be extended with extra CPU clusters. A SoC 'a_soc' can be extended like this: > socs: > extend: a_soc > cpuclusters: > - name: extra_core Signed-off-by: Torsten Rasmussen <[email protected]> (cherry picked from commit c602935faf814c161ea78d19e5a858f7022978c2) (cherry picked from commit e371648) (cherry picked from commit 5de335b)

…sion Upstream PR: zephyrproject-rtos/zephyr#72857 This commit adds new tests for verifying the functionality of the board and SoC extension feature. It does so by defining: - A new CPU cluster on an existing SoC - Two new board variants on top of an existing board The new board variants are defined on top of the existing `native_sim` board, so that the following native_sim board targets are available for the test. Existing board targets: - native_sim/native - native_sim/native/64 Extended board targets: - native_sim/native/one - native_sim/native/64/two The new CPU cluster is defined for the existing `an521` SoC. Existing CPU Clusters on an521: - cpu0 - cpu1 New CPU Cluster: - cputest For SoC tests the mps2 board is used. This means that for testing, the following board targets using the an521 SoC are: - mps2/an521/cpu0 - mps2/an521/cpu1 - mps2/an521/cputest Signed-off-by: Torsten Rasmussen <[email protected]> (cherry picked from commit 04dda379d7ea202712c68aba60b473cd4b5774ef) (cherry picked from commit e766f9f) (cherry picked from commit 0d8c22b)

…config Upstream PR: zephyrproject-rtos/zephyr#72857 Adds an optional priority parameter to the flash runner run once configuration which allows for deciding upon which file should ultimately be used Signed-off-by: Jamie McCrae <[email protected]> (cherry picked from commit 88a4c4ecf6906c7b66e6f42402f1f3e22456ff05) (cherry picked from commit c88231a) (cherry picked from commit 59371c3)

…n once config Upstream PR: zephyrproject-rtos/zephyr#72857 Removes validating the qualifiers for flash run once configuration as files may be present that contain information for qualifiers that are not present in a single repository but are spaced out in other repositories, or might be optional Signed-off-by: Jamie McCrae <[email protected]> (cherry picked from commit b72acf47e550090878fa87145c9766504c3deb13) (cherry picked from commit 973e62f) (cherry picked from commit 4f0db94)

Upstream PR: zephyrproject-rtos/zephyr#72857 In the `Board` class, the `dir` member was renamed to `directories`, both to indicate that it is now a list (in HWMv2 with board extensions) and to reflect the addition of the `BOARD_DIRECTORIES` CMake variable. Considering that the build system also keeps the `BOARD_DIR` variable, and for the sake of backwards compatibility and brevity, it should be useful to retain `Board.dir` in Python as well, symmetrically. Signed-off-by: Grzegorz Swiderski <[email protected]> (cherry picked from commit 25fd4b16e8c488792d0d8450e1b822292f94d3c3) (cherry picked from commit 423ed80) (cherry picked from commit 4175a49)

After zephyrproject-rtos#72857 the concept of board directory was expanded from a single path (BOARD_DIR) to a list of paths (BOARD_DIRECTORIES) which is taken into account in several places except when including `pre_dt_board.cmake` files. This patch moves the inclusion of `pre_dt_board.cmake` files into `dts.cmake` AFTER `boards.cmake` is included (so we can access `BOARD_DIRECTORIES`) but BEFORE `pre_dt.cmake`, which preserves the original inclusion order relative to pre DTC operations. The original motivation of this patch was to address the practicality of defining board extensions that can re-use base board definitions. In particular, one of the drawbacks of zephyrproject-rtos#72857 is that board extensions are not really extending board definitions just the qualifier. There is still the need to create a DTS for each new board extension which is often a copy-and-paste of the base board DTS eventually with a few alterations. The problem is that except for toy examples, board DTS files are rarely self-contained and instead are composed of multiple dtsi includes using relative paths which then forces board extension files to either use hard coded paths to the base board directory OR recursively copy all included dtsi files from the base board directory into the board extension directory. With this patch a board extension directory can define custom configuration steps like regular base boards can and then adjust CPP and DTC include search paths dynamically, for example with: ``` list(APPEND DTS_EXTRA_CPPFLAGS "-I${BOARD_DIR}") list(APPEND EXTRA_DTC_FLAGS "-i${BOARD_DIR}") ``` Signed-off-by: Nicolas Lebedenco <[email protected]>

tejlmand requested review from 57300, carlescufi and nordicjm May 16, 2024 08:59

zephyrbot added the area: Build System label May 16, 2024

zephyrbot requested review from jeremybettis and nashif May 16, 2024 08:59

zephyrbot assigned tejlmand May 16, 2024

tejlmand force-pushed the issues/69548_board_extension branch 2 times, most recently from 669ab34 to a9fbefe Compare May 16, 2024 12:38

zephyrbot added area: West West utility area: Documentation Infrastructure area: Continuous Integration area: Coding Guidelines Coding guidelines and style labels May 16, 2024

zephyrbot requested review from fabiobaltieri, gmarull, jfischer-no, kartben, keith-zephyr, mbolivar-ampere, stephanosio and swinslow May 16, 2024 12:38

tejlmand force-pushed the issues/69548_board_extension branch 2 times, most recently from 550d074 to d0dbcc4 Compare May 17, 2024 10:45

tejlmand mentioned this pull request May 22, 2024

Board-specific overlays defined in a module's shield directory are not found #71113

Open

nashif added the Architecture Review Discussion in the Architecture WG required label May 22, 2024

carlescufi removed the Architecture Review Discussion in the Architecture WG required label May 30, 2024

57300 reviewed Jun 5, 2024

View reviewed changes

tejlmand dismissed PerMac’s stale review via 891b895 October 25, 2024 13:15

tejlmand mentioned this pull request Oct 25, 2024

doc: extend board porting guide with new board extend feature #73409

Closed

kartben requested changes Oct 25, 2024

View reviewed changes

doc: extend board porting guide with new board extend feature

c145817

Extend the board porting guide with documentation on the new board extend feature. Signed-off-by: Torsten Rasmussen <[email protected]>

tejlmand force-pushed the issues/69548_board_extension branch from 891b895 to c145817 Compare October 25, 2024 16:50

henrikbrixandersen approved these changes Oct 25, 2024

View reviewed changes

kartben approved these changes Oct 26, 2024

View reviewed changes

carlescufi requested review from PerMac, nashif and nordicjm October 26, 2024 09:59

carlescufi approved these changes Oct 26, 2024

View reviewed changes

nordicjm approved these changes Oct 26, 2024

View reviewed changes

57300 approved these changes Oct 26, 2024

View reviewed changes

nashif reviewed Oct 26, 2024

View reviewed changes

nashif approved these changes Oct 26, 2024

View reviewed changes

henrikbrixandersen merged commit c4fcf56 into zephyrproject-rtos:main Oct 26, 2024
43 checks passed

tejlmand mentioned this pull request Nov 6, 2024

shield: m5stack m5dial #80778

Closed

tejlmand mentioned this pull request Nov 25, 2024

Hardware Model v2 - Ability to extend an existing board #69548

Open

nlebedenco mentioned this pull request Jul 20, 2025

cmake: include pre_dt_board.cmake from all board directories #93393

Open

	soc_folders = {soc.folder[0] for soc in v2_systems.get_socs()}
	soc_folders = {folder for soc in v2_systems.get_socs() for folder in soc.folder}

Support SoC and board extensions #72857

Support SoC and board extensions #72857

Uh oh!

Conversation

tejlmand commented May 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nashif commented May 22, 2024

Uh oh!

henrikbrixandersen commented May 28, 2024

Uh oh!

PerMac commented May 29, 2024

Uh oh!

57300 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tejlmand commented Oct 25, 2024

Uh oh!

carlescufi commented Oct 25, 2024

Uh oh!

kartben left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tejlmand Oct 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

henrikbrixandersen left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kartben left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nashif left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

tejlmand commented May 16, 2024 •

edited

Loading

tejlmand Oct 25, 2024 •

edited

Loading

henrikbrixandersen left a comment •

edited

Loading