build: expand ${ZEPHYR_<module>_MODULE_DIR} in CONF_FILE, OVERLAY_CONFIG

[gregshue]

Support referencing module directories by name in CONF_FILE, and
OVERLAY_CONFIG so that projects can reference overlay files in arbitrary
modules.

Fixes #41830

Signed-off-by: Gregory Shue [email protected]

[tejlmand]

Thanks for this proposal.

I see the usefulness in it, but some general thoughts before going into implementation details.

To be consistent between specifying configuration files, then DTC_OVERLAY_FILE should also be updated to have similar behavior.

One concern with this approach is that it will be possible to use any CMake variable, for example -DCONF_FILE=<path>/${FOO}.conf.
If users start to depend on specific variables inside the build system being defined before the CONF_FILE is processed, then we will make it harder for ourselves to refactor our internals, cause suddenly some users might be depending on FOO being defined before CONF_FILE is used. FOO could even be an internal variable.

Of course, some variables could be considered safe, as they will most likely always be available before CONF_FILE is used.
For example: ZEPHYR_<name>_MODULE_DIR and BOARD.
As minimum it should be clearly documented what variables we officially support with this approach.

So please update the documentation to describe this new possibility, together with officially supported variables that can be used.

[gregshue]

Thank you. Do you have a doc section in mind for this content?

[tejlmand]

Thank you. Do you have a doc section in mind for this content?

As general description, maybe adding it here:
https://docs.zephyrproject.org/latest/application/index.html#application-configuration-directory
for example as continuation of:

Zephyr will use configuration files from the application’s configuration directory except for files with an absolute path provided by the arguments described earlier, for example CONF_FILE, OVERLAY_CONFIG, and DTC_OVERLAY_FILE.

For CONF_FILE I would suggest we also adding this knowledge here:
https://docs.zephyrproject.org/latest/guides/build/kconfig/setting.html#the-initial-configuration

All configuration files will be taken from the application’s configuration directory except for files with an absolute path that are given with the CONF_FILE argument.

with reference to variables that are officially supported with this method.

For devicetree overlays, here:
https://docs.zephyrproject.org/latest/guides/dts/howtos.html#set-devicetree-overlays

[tejlmand]

Thanks for updating the docs.

Some comments, to ensure it become more precise.

[tejlmand]

Thanks for the update.

I took a look at the rendered output, and have a small proposal.

[tejlmand]

lgtm, thanks for this.

Found a stray the to be removed before merging, but approving as everything else is fine.

[tejlmand]

lgtm - re-approved

[github-actions]

This pull request has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this pull request will automatically be closed in 14 days. Note, that you can always re-open a closed pull request at any time.

[tejlmand]

@gregshue seems this has gone stale.
If you still would like it in, please re-open and rebase, then i'll re-approve

[gregshue]

@tejlmand I would still like this in, but I have not recognized any option to reopen the pull request. (I looked at https://github.community/t/re-opening-pull-request/1337/2 for guidance, but the "Reopen pull request" button shown next to "Comment" is not present in my view.)

Guidance?

[dleach02]

@nashif, are you able to reopen this? The button is greyed out for me

[gregshue]

@nashif Are you able to reopen this? I'm not given the option.

[gregshue]

@tejlmand @dleach02 @nashif Since we are having trouble re-opening this PR, I created a new one with the same changes (#47638)