Optional deps

[benjeffery]

No description provided.

[benjeffery]

I've made all the format-specifc deps including cyvcf2 optional.
Needs some concrete CI tests for the functionality.

[jeromekelleher]

Nice - so the idea is that if you just need to convert plink, you don't have to install cyvcf2? Infrastructure LGTM.

I have some vague recollection of there being a recent PEP on handling this sort of thing, but haven't managed to track it down. @tomwhite, do you know what current best practices on handling optional deps is?

[benjeffery]

so the idea is that if you just need to convert plink, you don't have to install cyvcf2

Exactly - cyvcf2 isn't an "easy" dep so seemed wise.

Maybe you are thinking of https://peps.python.org/pep-0631/ ??

[tomwhite]

I have some vague recollection of there being a recent PEP on handling this sort of thing, but haven't managed to track it down. @tomwhite, do you know what current best practices on handling optional deps is?

Possibly this one? https://peps.python.org/pep-0771/

Optional packages can cause problems with conda (as in conda doesn't have a way to install them), but I'm not sure what the best practices there are.

[benjeffery]

As far as I'm aware there in conda-land the "best" option is to have several packages e.g. bio2zarr-all bio2zarr-plink ... but that isn't great. Prob best just to have one big build.

[jeromekelleher]

Optional packages can cause problems with conda (as in conda doesn't have a way to install them), but I'm not sure what the best practices there are.

I think you just have to do one big build in conda - I guess the good thing there is that you find out whether that's actually possible as you're building the package. Let's just build for pip for now, and worry about conda later.

[jeromekelleher]

Aha, yes, thanks Tom - it was pep-771. So, we'd default to installing the VCF version if someone requests bio2zarr without any extra specifiers. Is that possible here?

[tomwhite]

Aha, yes, thanks Tom - it was pep-771. So, we'd default to installing the VCF version if someone requests bio2zarr without any extra specifiers. Is that possible here?

I'm not sure it is until 771 is adopted - I think we fall into https://peps.python.org/pep-0771/#packages-supporting-multiple-backends-or-frontends, where "installing the package without any extras results in a broken installation". So for now we just have to document the installation process clearly.

[coveralls]

coverage: 98.166% (+0.01%) from 98.153%
when pulling 06f8e30 on benjeffery:optional-deps
into 587a29e on sgkit-dev:main.

[jeromekelleher]

LGTM

[jeromekelleher]

Let's keep the comments here with the rationale for these dependencies. The only reason we're using coloredlogs is because it comes anyway with cyvcf2

[benjeffery]

I've added the new CI job to the required merge tests.