Automatic code setup: validation and --all option

[t-reents]

Addressing parts of #1156

• adding validation
• --all option. To make this more convenient, I also added the --ignore-missing and --skip-existing options. I agree with you @mbercx, that I'd try to keep this as minimal as possible. I figured that --all (and the related ones) make sense. However, I don't really see the need for a dry-run.

Since we have the --all option now, the executables argument is optional now.

Even without the new --all option, adding a --ignore-missing makes sense. I just realised that the command would fail when specifying three executables of which one is missing, which was a bit annoying.

[eimrek]

maybe good to also add pw2wannier if available? needed for wannier workflows.

[t-reents]

@eimrek Ah thanks! I remember that I actually typed it and some point, but it got somehow lost.

[mbercx]

Thanks @t-reents! I only had a quick look so far, will give a proper code review when we decide on the API. I left one comment with some questions that are slightly beyond the scope of this PR (apologies), but definitely related, and I think will influence the decision that is made here.

I'm still not entirely convinced that adding --all is a good idea. I think the benefit for the user is rather minimal. Probably a lot more time will have to be spent on thinking about the right prepend_text etc than typing the codes you need. Here are some downsides that I see:

1. If you add the forgotten open_grid.x, we'll already be setting up 13 codes. d3hess.x and average.x are being added in #1225 and #1226, respectively. If we decide to also support QE executables that are not in this package (also see my comment), we might be looking at close to 20 codes being set up from which a typical user might 3 or 4. Doing verdi code list is going to give quite a few codes. 😅 Of course, it's up to the user, but I'm not sure how many users would really be happy with having all these codes set up, and I don't know if we should encourage such messy configurations.
2. Added complexity. This is both in the API (where users suddenly get three more options to consider), and our code, which has to deal with all these options and their combinations. The more options/inputs we have, the more careful I am with adding more. It should really be worth it. It's possible this argument is less important if there are standalone reasons to keep the two other options and I have a closer look at the code.

[mbercx]

I think you also forgot open_grid.x here. :)

One issue with this approach is that (besides forgetting to update this list), there might be other codes that are supported in other plugin packages that aren't in this list, but where I think our code setup command would be able to set up the code just fine (e.g. hp.x, possibly others). I suppose we could add those here as well, but then the list for --all becomes even longer and we're adding logic for external packages that we don't depend upon.

I was also going to add epw.x and xspectra.x here as well, but then I realised that their entry points are epw.epw and xspectra.xspectra. So sadly I won't be able to use our beautiful command to set up these codes. 🥲 I'm wondering if it would be worth considering supporting these executables still somehow (e.g. via a mapping), or it's also too messy because they are external non-dependencies.

If we keep this validation, we should adapt the language though: e.g. use "supported" executables instead of "valid". xspectra.x is a perfectly valid QE executable. I'm even a bit worried that for users of Xing's plugin package this won't be confusing: why can't I install xspectra.x with this command?

[mbercx]

Even without the new --all option, adding a --ignore-missing makes sense. I just realised that the command would fail when specifying three executables of which one is missing, which was a bit annoying.

Fair point! Some questions here:

1. Did the command fail because you wrote the wrong executable? Because for typically QE installations I would expect all our supported executables to be there.
2. Instead of adding an --ignore-missing option, should raise a warning instead of an error by default?
3. Now any issues with missing executables are entirely silences with ignore_missing = True. Is that a good idea?
4. Should we also ignore missing executables in case the directory is specified?

[t-reents]

@mbercx Finally getting back to it.

I just read your comments again, and I also think that --all might not be the best option. At least personally, I would never use it, as I don't want to see ~20 executables for each computer :D Also, I don't see the use-case where someone is really interested in using all the QE executable in a single AiiDA profile/project.

Moreover, it will simplify the implementation.

Finally, something that came to my mind, what about adding a sensible default, as a kind of trade-off between the two extremes (--all option or always explicitly specify the executables). I was thinking of pw projwfc dos. Having in mind the motivation for this command, to make it very simple for us but especially also new users, I think that those are reasonable and the ones that are covered in any AiiDA tutorial. What do you think?

[t-reents]

Instead of adding an --ignore-missing option, should raise a warning instead of an error by default?
Now any issues with missing executables are entirely silences with ignore_missing = True. Is that a good idea?

Indeed, probably not such a good idea. Thinking about it, I actually like the first option, e.g. using a warning. Nonetheless, one advantage of raising by default is that it is very explicit, i.e. users will immediately realize that something went wrong. However, one will probably also realize quickly that the code is missing and look at the warning at that point

[mbercx]

Thanks @t-reents!

Re the default: so you are suggesting to make the executables argument optional? Again: I'm not sure if this helps more than it hurts. Now the API is very clear: you pass the executables you want to install. Having to explain that by default it will instal X, Y and Z, and dealing with all that.. I don't think it's worth it. Is it really that hard to add the executables? The command is also already in the documentation and can be copied with a single click.

Re --ignore-missing: I think a warning instead of raining is sufficient, still allows installation of the other codes, and the user should be able to easily notice the warning and indeed absence of the code. Have to dogfood a little and refresh the command UX, but I would vote for that.