jaxlib removal from pypi

[vklogin]

I noticed that jaxlib 0.1.38 was removed from pypi earlier this week. Per https://pypi.org/project/jaxlib/#history the oldest version is now 0.1.41. What is the plan for removal of future versions from pypi?

Thank you

[hawkinsp]

We're out of space on PyPi. We will remove old releases whenever we make new releases and find we don't have space.

[hawkinsp]

Well... we're close to the 10GB limit. Although we deleted a few releases this time around. I'd suspect we'd next need to delete a few more in a month or two.

If it is helpful, we can try to keep copies of all of our wheels in our GCS bucket (which is where the GPU wheels are kept).

[yashk2810]

The wheels are already up on GCS buckets.

Examples: (Please change the python version/CUDA version per your setup)

For CPU jaxlib wheels you can do:

pip install https://storage.googleapis.com/jax-releases/nocuda/jaxlib-0.1.70-cp39-none-manylinux2010_x86_64.whl

For Cuda jaxlib wheels you can do:

pip install https://storage.googleapis.com/jax-releases/cuda110/jaxlib-0.1.70+cuda110-cp37-none-manylinux2010_x86_64.whl

[matthewfeickert]

@yashk2810 Out of curiosity, will these wheels in the GCS buckets be supported indefinitely? Or is there a plan to eventually drop them? I ask given that pyhf has a "Minimum supported dependencies" workflow in which we install the lowest bounds of all our dependencies to run the tests. We currently have in our lower-bound-requirements.txt

...
# jax
jax==0.2.8
jaxlib==0.1.58

which is now failing. If I throw in

...
# jax
--find-links https://storage.googleapis.com/jax-releases/jax_releases.html
jax==0.2.8
jaxlib==0.1.58

this should work again, but I just wanted to have some idea if for testing stability purposes it is better to view these releases as basically dead.

[yashk2810]

I just deleted 0.1.58 because of storage issues. The wheels in GCS should be present indefinitely.

[matthewfeickert]

I just deleted 0.1.58 because of storage issues.

Yup, that's what I assumed.

The wheels in GCS should be present indefinitely.

Great! Thanks for the super speedy info. 👍