Patch release 0.18.1 #2983
Merged
Patch release 0.18.1 #2983
+57
−18
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
With the v5 rc being out, we should now ensure that the PEFT tests pass. This PR contains fixes to achieve that. 1. hub_online_once was failing because transformers.utils.hub._is_offline_mode no longer exists. Using the new function instead if transformers v5 is detected. 2. tests/test_encoder_decoder_models.py::TestEncoderDecoderModels::test_merge_layers[LoraConfig-config_kwargs10-peft-internal-testing/tiny-random-BartForConditionalGeneration] failing due to TrainableTokensWrapper not being applied to all layers owing to changes to _tied_weights_keys. 3. While working on this, I discovered a tangential bug in TrainableTokensLayer.get_merged_weights. This method returns a torch.Tensor but the expected type is nn.Parameter (since foo.bar.weight is supposed to be a nn.Parameter). This type mismatch would cause torch's model.get_parameter, which I used in _get_module_names_tied_with_embedding, to fail. At first, I wanted to change the return type to nn.Parameter but this causes all kinds of issues. Therefore, I left this bug as is. Instead, in _get_module_names_tied_with_embedding, I opted to use attrgetter instead of model.get_parameter.
E.g. it's not available for the torch rocm build. Signed-off-by: vladmandic <mandic00@live.com>
Resolves huggingface#2975 In huggingface#2826, we inadvertently added a dependency on transformers v4.52 to PEFT. However, this is really only needed under very specific circumstances (aLoRA + gradient checkpointing). With this PR, unless we're in these circumstances, this requirement is no longer there.
Contains the following changes: - huggingface#2934 - huggingface#2963 - huggingface#2976
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
BenjaminBossan
merged commit e3398fc
into
huggingface:release/v0.18.1
8 of 13 checks passed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Contains the following changes: