question about api request flow to use https://prefix.dev/$channel as conda channel

[trim21]

I'm currently working on adding pixi support for https://github.com/renovatebot/renovate, which is a dependency automation tool. I have implement the support of using channel on anaconda server.

There is a need to get versions of a package. With conda channel it's possible to use api like https://api.anaconda.org/package/conda-forge/numpy/ , but it's also possible to define https://prefix.dev/conda-forge in pixi.

So I read the docs of prefix.dev, and I understand I can use api from https://prefix.dev/api/graphql, but how do I know then I should translate a http url in channels looks like there is no reliable way to know if a conda conda register support some kind of API?

It should be best to avoid using repodata.json, I even got OOM error with noarch/repodata.json on my desktop browser 😅

[trim21]

OK, I know, https://prefix.dev/conda-forge/ is a real channel and you can use https://prefix.dev/conda-forge/noarch/repodata.json and everything else here, but not json api like api.anaconda.org only a GraphQL API.

It's not very easy to support it, this require a new data source in renovatebot. If prefix.dev has a compatible with api.anaconda.org it would be much easier 😅

[pavelzw]

Exactly. A channel always consists of noarch/repodata.json and then for every architecture it supports <arch>/repodata.json (the most relevant channels are linux-64, osx-arm64, osx-64, linux-aarch64, linux-ppc64le and win-arm64).
This is the minimum that a conda channel must support for it to be accepted by conda/mamba/pixi.

conda-forge on anaconda.org for example also supports repodata.json.zst or repodata.json.bz2 and repodata.json.jlap to speed downloads up.
The conda-forge mirror on prefix.dev supports the things mentioned before as well as sharded repodata (see https://prefix.dev/blog/sharded_repodata and conda/ceps#75 for more info) which is a faster way to download the relevant parts of the repodata.
Rattler should have functions implemented to fetch these repodatas in the rattler_repodata_gateway crate and select the fastest one by default. These functions should IMO be exposed to rattler-js that you can then use.

I would discourage from using APIs from anaconda.org or graphql APIs from prefix.dev since they are not a well-defined standard and this would also lead to your updating mechanism breaking for things like self-hosted conda channels inside corporate infrastructure that only support the repodata.json method.

[trim21]

the most relevant channels are linux-64, osx-arm64, osx-64, linux-aarch64, linux-ppc64le and win-arm64

Why no win-64?

[pavelzw]

Forgot to mention it

[wolfv]

@trim21 thank you so much for the work you are doing! One could indeed use the graphql API that we expose on prefix.dev and that would be fine. However, @baszalmstra also has worked on integrating the "repodata gateway" into the rattler JS bindings. With this gateway it is possible to query the repodata directly (the same way pixi would do). We also support "sharded repodata" which means you'll only get the index data for e.g. numpy or python instead of the very large repodata.json that contains information for all packages in the channel.

Maybe @baszalmstra has additional thoughts on this but I think that will be the best way.