Skip to content

Fix the 'off by 1' problem in dynamically resized LoRA rank#2194

Open
woct0rdho wants to merge 1 commit intokohya-ss:mainfrom
woct0rdho:rank1
Open

Fix the 'off by 1' problem in dynamically resized LoRA rank#2194
woct0rdho wants to merge 1 commit intokohya-ss:mainfrom
woct0rdho:rank1

Conversation

@woct0rdho
Copy link
Contributor

@woct0rdho woct0rdho commented Sep 4, 2025

If I understand correctly, the index returned by index_sv_cumulative and index_sv_fro is larger by 1 than it should be. For example, torch.searchsorted works like this:

>>> cumulative_sums = torch.tensor([0.9, 0.95, 1.0])
>>> print(torch.searchsorted(cumulative_sums, 0.94))
tensor(1)
>>> print(torch.searchsorted(cumulative_sums, 0.95))
tensor(1)
>>> print(torch.searchsorted(cumulative_sums, 0.96))
tensor(2)

If the user wants to keep 94% or 95% of S, then index should be 1 (and new_rank in the function rank_resize should be 2).

This problem is particularly visible when the new LoRA is rank 1 (I've seen many LoRAs on the internet that are almost rank 1). In the example above, if the user wants to keep 80% or 90% of S, then index should be 0, and new_rank should be 1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant