Simulate the behavior of pip's --extra-index-url

[bruno-brant]

I'm trying to install a CUDA distribution of pytorch, and the documentation asks me to use a extra index:

pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116

I tried simulating this behavior by adding a additional source to Poetry:

poetry source add cu116 https://download.pytorch.org/whl/cu116 --secondary

However, Poetry began to try to download everything from there, instead of just torch and torchvision.

Is there a way to replicate this behavior?

[neersighted]

This is currently the intended behavior of secondary sources. They were mostly an afterthought not originally intended for the 'specific packages from this source use case.' We are actively discussing new designs around repository searching, and how to move forward in a backwards compatible way.

Please see my proposal at #5984 (comment) and offer any input you have there -- we will likely migrate discussion to a new PR or issue if the original author does not want to see an agreed upon design through, but for now that is the correct venue.

[neersighted]

I suppose it's also worth noting that a non-secondary source is actually the equivalent to --extra-index-url -- pip just doesn't say anything when it tries an index for a package and fails, and is much faster than us as they do not gather package metadata in their resolver.

[bruno-brant]

Hello @neersighted, thanks for the response. I'm not that versed in pip, good to know know. That means if I set it up correctly I can get the behavior I want.

I'll try to contribute to the proposal. I think Poetry is a must in the Python ecosystem. I migrated a number of internal projects in my org from stupid requirements.txt to it.