Skip to content

Conversation

@tomaarsen
Copy link
Member

@tomaarsen tomaarsen commented Oct 10, 2024

Hello!

Pull Request overview

  • Increment Sentence Transformers version to v3.1.1

Details

Version v3.0.1 is required to load models that have Dense modules (https://huggingface.co/sentence-transformers/LaBSE) with Safetensors only. Currently, with v2.7.0, any model with a Dense module that also has Safetensors files, will likely fail in Inference Endpoints.
The reason is that the pytorch_model.bin files won't be copied if any model.safetensors file is detected, so the Dense module won't be able to load as we need v3.0.1 for that.
See also this Slack thread from a while ago for more details.

This is also preventing me from adding Safetensors files to LaBSE.

cc @philschmid

  • Tom Aarsen

Version 3.0.1 is required to load models that have Dense modules (https://huggingface.co/sentence-transformers/LaBSE) with Safetensors only.
@philschmid philschmid merged commit 03fcc58 into huggingface:main Oct 10, 2024
6 checks passed
@tomaarsen
Copy link
Member Author

@philschmid I'm not sure what the connection between huggingface-inference-toolkit and the Inference Endpoints are - when can I expect to be able to add safetensors to LaBSE without breaking inference endpoints for people?

  • Tom Aarsen

@philschmid
Copy link
Contributor

@ErikKaum should be able to help here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants