Load `west_commands.yml` from default locations

[thorsten-klein]

Currently west extensions can only be loaded via a west_commands.yml list that a west manifest points at.

For users it could make sense to support also some default locations, e.g.:

1. Workspace Commands from .west/west-commands.yml
2. System-wide Commands: ~/.config/west/west-commands.yml
3. User commands via config option west.commands_ymls (respectively some env variable like WEST_COMMANDS_YMLS

For 2. and 3. even no west workspace could be necessary.
This would make west really flexible, e.g. users could start using west as a simple command wrapper or frontend even if they do not use west workspaces at all.

What's your opinion on this?

[pdgendt]

If anything it should be configured/guarded by a config option, to make it opt-in.

[marc-hb]

Why not, what specific use cases do you have in mind?

The WestCommand constructor already has a requires_workspace parameter.

If anything it should be configured/guarded by a config option, to make it opt-in.

BTW there is already a commands.allow_extensions boolean.

cc:

• @mbolivar
• Clean way to import a west.yml's west commands but no submodule #903

[yvesll]

For 2. and 3. even no west workspace could be necessary.

This feature would be very useful for me. I have multiple west workspaces locally, and I want to add some common extensions that do not depend on a specific workspace. Previously, I had to port these extensions into every repository, and whenever there was an update, I had to update each one individually.

[henrikbrixandersen]

For 2. and 3. even no west workspace could be necessary.

This feature would be very useful for me. I have multiple west workspaces locally, and I want to add some common extensions that do not depend on a specific workspace. Previously, I had to port these extensions into every repository, and whenever there was an update, I had to update each one individually.

Have you considered putting these extensions in a share repository brought in via each west.yml?

[yvesll]

Have you considered putting these extensions in a share repository brought in via each west.yml?

Yes, some extensions were added in a separate repo. But it doesn’t solve all difficulties.
This approach requires a west workspace and developer has to modify the west.yml or add a new one to pull the shared repo in. For scenarios where there is no workspace (e.g., using west purely as a command wrapper), it doesn’t work.

[thorsten-klein]

@yvesll seems to have the same use case as we have. Thank you!

If I understand you all correctly, I should focus only on the third approach to opt-in via west config or env variable (where users can specify one or more additional extension files). That's totally fine and perfectly enough for our use case 👍

[pdgendt]

If I understand you all correctly, I should focus only on the third approach to opt-in via west config or env variable

I'd advice against using an env variable for this.

[yvesll]

If I understand you all correctly, I should focus only on the third approach to opt-in via west config or env variable (where users can specify one or more additional extension files). That's totally fine and perfectly enough for our use case 👍

I prefer local and global west config option.

[thorsten-klein]

I'd advice against using an env variable for this.

If users want to enable the extensions globally, they set the according config option (e.g. in the global or system config). But what if a local workspace sets the same config option? Then it will override the global or system one.
Therefore I also thought about some other opt-in mechanism like env variables.

[pdgendt]

If users want to enable the extensions globally, they set the according config option (e.g. in the global or system config). But what if a local workspace sets the same config option? Then it will override the global or system one. Therefore I also thought about some other opt-in mechanism like env variables.

You can change the application logic to accumulate the options instead

    key = 'commands.extra' # or something better
    paths = (
        f"{config.get(key, default="", configfile=ConfigFile.LOCAL)};"
        f"{config.get(key, default="", configfile=ConfigFile.GLOBAL)};"
        f"{config.get(key, default="", configfile=ConfigFile.SYSTEM)}"
    )

EDIT: Or a custom placeholder that can be put in the list to denote to load from global/system too. This would allow to override if desired.

[marc-hb]

You could give each config level the ability to point at a "dropins" directory west_commands.d/*.yml. Then you just load all the .yml files in all the dropins directory in a well defined order. Fatal error if an extension gets redefined?

I just looked at zephyr/scripts/west_commands.yml and that file looks unnecessary, probably a design mistake. The little information it has could/should be part of the .py files themselves. Funny enough its first line says this:

# Keep the help strings in sync with the values in the .py files!

So there's probably a good opportunity to drop the intermediate west_commands.yml file in this case and increase the flexibility.

EDIT: the rationale is documented, my bad. Maybe it could still be optional.

Previously, I had to port these extensions into every repository, and whenever there was an update, I had to update each one individually.

Thanks, great justification: unwanted copy/paste/diverge is always very bad.