cmake: dts.cmake: Add Board overlays before shields

[JonathanVanWin]

Is your feature request related to a problem? Please describe.
I want to be able to configure the board peripherals before the shield. If my overlay configures the peripheral in one way, and the shield in another, I want the default configuration to come from the overlay, and then the shield will make the changes on that. Currently it processes the shield before, and then the overlay, overriding the shield configurations.

Describe the solution you'd like
It should process the board dts (DTS_SOURCE) firstly. Then the board overlay (add a new cmake variable such as DTC_BOARD_OVERLAY), then the shields, and lastly all the other overlays (DTC_OVERLAY_FILE).

Describe alternatives you've considered
There is no good alternative, because if I don't want shields to be overridden, I should have the same peripheral configuration also in the board overlay, and that removes the meaning of a shield.

[tejlmand]

@JonathanVanWin we need to be very careful here.

Please take a look here also: #29074

You may have board specific overlays at multiple levels, the sample itself, the shield folder, and then user specified overlays with DTC_OVERLAY_FILE.

When you refer to a board overlay, I assume you mean the board overlay in the samples folder and you want that sourced BEFORE the shield overlay is sourced, is that correctly understood ?
Or do you actual mean that you want an extra argument, as to control when an overlay is applied ?
Like this: cmake -DDTC_BOARD_OVERLAY=foo.overlay -DDTC_OVERLAY_FILE=bar.overlay -DSHIELD=myshield ?

Where foo.overlay will be applied in the middle of the chain.

so far, i'm not convinced this is a good idea.
Currently

• board dts is sourced
• shield specific overlay
• board specific overlays defined by the shield
• sample specific overlay (app.overlay / board.overlay)
• DTC_OVERLAY_FILE

That is a clear order, and I would like to know if you have looked into having a board specific overlay for the shield(s) you're working with, similar to the sparkfun shield: https://github.com/zephyrproject-rtos/zephyr/tree/main/boards/shields/sparkfun_sara_r4/boards

Could you perhaps explain why a board specific overlay in the shield folder is not working in your case.

[JonathanVanWin]

Okay, I will try to explain how we work.

• We have an infrastructure wrapping zephyr, that supports different projects.

• Every project normally runs on one type of SoC, but can have different boards: an ev board to test it, and the "real" board.

• Because we use the same boards on different projects, and we don't need the stuff defined in the zephyr board, we have a board "reset", desabling by default all enabled peripherals, and we add our own flash partitions we normally use, default configurations for common peripherals, and chosen nodes (this is for the ev boards we use for testing). This file is our .overlay (and it is not located in the samples directory, we have it in our boards one).

Like I explained before, this file has default configurations for our peripherals, but we may not want for every project the same configuration, so we want to override it if needed.

• Because we use more than one board (normally same SoC) for one project, we want to have the common hw configurations in the same file, and for that we use the shields. We set the chosen nodes, aliases and specific peripheral configurations, that are custom to this project in this file. If we use more than one board for this project in the shields we have also the boards dir overriding the board pins. (BTW untill here all our peripherals are disabled).
• Then we have what you call the app/board.overlay, we only use this to enable only the peripherals we use.

You also need to know, that all the dts file handling we do on our own cmake file: before including the boilerplate cmake file, we set our cmake configurations and variables, and here we set the DTC_OVERLAY_FILE, SHIELD and OVERLAY_CONFIG variables.

The only part we don't have control is the shield part, where it gets included before our .overlay (see above for explanation, it is not the sample dts one, we have our own way of dealing with hw files).

Because the shields get before our own board overlay, the default board configurations we have there to have out of the box configured peripherals, override our specific project configurations (that are inside the shields file), such as baud rate or different pin configurations.

I don't see how my proposition would break zephyr, because it is only adding a new variable we can use to include a dts file before the shields one, and after the zephyr board dts one. If not used, the order would still be the same, and adding it, would solve our problem letting us have our .overlay sourced before the shields, letting the shields file override the default configurations with specific project ones.

Could you perhaps explain why a board specific overlay in the shield folder is not working in your case.

The problem is that we don't always want to use a shield. For e.g., if we make a sample (not a project), we may not want to create a shield just for that sample, and want to use instead the board default configurations, and enable them in the sample overlay.
And we may also not want to just use the shield, because not all the configurations there are appropriate just for a small sample.
Therefore it's easier to have the default ones, and override them with shield for project.

[tejlmand]

Every project normally runs on one type of SoC, but can have different boards: an ev board to test it, and the "real" board.

sure, quite common.
And you can share dts files between boards, for example like this:

zephyr/boards/arm/nrf9160dk_nrf9160/nrf9160dk_nrf9160.dts

Line 9 in 35ec3c9

#include "nrf9160dk_nrf9160_common.dts"

Which is shared between the nrf9160dk_nrf9160 and nrf9160dk_nrf9160_ns boards.

Now, this will of course result in dedicated overlay configs in a sample foo if you want to adjust the same setting for myboardA and myboardB.

An alternative to this could be to create board variants (board revisions).
That way you may have:
myboard@real and myboard@test.
That will allow you sample to do:

<sampleA>
  |--- boards
        |--- myboard.overlay (used by both `@real` and `@test`)

or dedicated overlays

<sampleB>
  |--- boards
        |--- myboard_real.overlay (used only by `myboard@real`
        |--- myboard_test.overlay (used only by `myboard@test`)
        |--- myboard.overlay (used by both in conjuction with above overlay)

Then you build as:

$ cmake -DBOARD=myboard@test ...

See more https://docs.zephyrproject.org/latest/guides/porting/board_porting.html#custom-revision-cmake-files
https://docs.zephyrproject.org/latest/guides/dts/howtos.html#set-devicetree-overlays

Note, this talks about board revisions, but a revision is basically just a specific form of a variant.

Because we use more than one board (normally same SoC) for one project, we want to have the common hw configurations in the same file, and for that we use the shields. We set the chosen nodes, aliases and specific peripheral configurations, that are custom to this project in this file. If we use more than one board for this project in the shields we have also the boards dir overriding the board pins. (BTW untill here all our peripherals are disabled).

Let me see if I understand this correctly.
You are creating a shield per sample ?
So that with sampleN there also exists a shieldN ?
This sound like a misuse of the shield concept, probably you would like to join this discussion: #40669

Instead of the shield, why not create a common overlay, and then include this in the specific board overlay, like this:

<sampleN>
  |--- boards
        |--- n_common.dts
        |--- myboardA.overlay
        |--- myboardB.overlay
        |--- ...

and then a single line in each boardA.overlay, boardB.overline, .... file like this:

#include "n_common.dts"

that way you of course has multiple files, but they all include same common dts file.

Combining this with the board variant I described above, then you might even not need multiple files, as all variants of a board can share the same overlay file directly.

Not to mention that in the few cases where a variant needs custom adjustments you simply create a myboard_test.overlay in addition. And that variant specific overlay file will be sourced exactly at the right location according to your need from what I understand.

I think you should look at those suggestions I have provided.

From what I understand, this particular request originates from a misuse of the shield concept, and not using the infrastructure already in place in an optimal manner.

Thus at current time I don't believe this feature request should be implemented.
It is also believed that #40669 addresses part of this feature request.

[jfischer-no]

Thus at current time I don't believe this feature request should be implemented.
It is also believed that #40669 addresses part of this feature request.

#40669 is far from being as good as current Shield support, in fact it makes no sense at all, just compare it with each other. I think we should rename shield support, for example to brick, and derive config-brick (common configuration for something) and hardware-brick (shields) from it.

[tejlmand]

#40669 is far from being as good as current Shield support,

Discussions regarding #40669 should be taken there.
Let @JonathanVanWin decide if that PR meets his needs or not.
And if he has comments / proposals to that PR, let him place it there.

[tejlmand]

closing, as this feature request has been tried before here:
#27901
and reverted here due to the consequences:
#29074

Most likely the snippet RFC #40669 will provide the needed functionality, else feel free to open a new feature request with more precise description as to what is needed and ensure the request is not breaking existing design.