add no_libucx matrix option in dependencies.yaml

[trxcllnt]

This option allows generating dependencies without libucx in the dependencies list, which is something we have to do for NVAIE/DLFW builds.

[pentschev]

I'm not sure I understand this, it allows installing packages that do not contain libucx as a dependency but those packages are built externally (i.e., not available in CI here), is that it? Furthermore, I thought NVAIE only used wheels, why do we need to also add this exception to conda dependencies?

[jameslamb]

I agree with this, and having this "early return" style at the top of the dependencies groups is the clearest way to do it.

To answer your questions @pentschev ...

allows installing packages that do not contain libucx as a dependency but those packages are built externally (i.e., not available in CI here), is that it?

Yes, exactly. In NVAIE builds we prefer system-installed libraries for UCX to those from wheels.

I thought NVAIE only used wheels, why do we need to also add this exception to conda dependencies?

It does only use wheels. I'm guessing @trxcllnt updated the conda-only lists too for the benefit of the open-source RAPIDS devcontainers, which can pull in a system install of UCX via https://github.com/rapidsai/devcontainers/tree/main/features/src/ucx

[trxcllnt]

I can delete it from the conda entry, but I thought it prudent to include for consistency. If we generate a dependency file with no_libucx=true, it's because we are using some UCX not available via conda or pip. NVAIE uses a special HPC-X build of UCX, so we can't have the libucx wheel in the env as well.

[jameslamb]

I support keeping it there for conda too.

[jameslamb]

/merge