[quantization] Suppport convolutions in SensitivityCalibrator#581
Merged
mhs4670go merged 1 commit intoSamsung:mainfrom Mar 25, 2026
Merged
[quantization] Suppport convolutions in SensitivityCalibrator#581mhs4670go merged 1 commit intoSamsung:mainfrom
convolutions in SensitivityCalibrator#581mhs4670go merged 1 commit intoSamsung:mainfrom
Conversation
stamalakhov
commented
Mar 25, 2026
| if show_progress is True: | ||
| print("Computing calibration set") | ||
| for prompt in tqdm.tqdm(dataset, disable=not show_progress): | ||
| if isinstance(prompt, torch.Tensor): |
Contributor
Author
There was a problem hiding this comment.
Let's process multiple inputs as well.
stamalakhov
commented
Mar 25, 2026
Comment on lines
+136
to
+141
| self.calibrated_types = [ | ||
| torch.nn.Linear, | ||
| torch.nn.Conv2d, | ||
| torch.nn.Conv1d, | ||
| torch.nn.Conv3d, | ||
| torch.nn.ConvTranspose2d, |
Contributor
Author
There was a problem hiding this comment.
Calibrate also convolutions.
stamalakhov
commented
Mar 25, 2026
| inp_ids = inputs.view(-1, inputs.shape[-1]) | ||
| logits = model(inp_ids.to(model.device)).logits | ||
| if isinstance(inputs, torch.Tensor): | ||
| inp_ids = inputs.squeeze(0) # remove redundant batch dimension |
Contributor
Author
There was a problem hiding this comment.
remove redundant batch dimension, instead of reshaping input to 2D shape.
stamalakhov
commented
Mar 25, 2026
Comment on lines
+173
to
+176
| for item in inputs: | ||
| inputs[item] = inputs[item].to(model.device).squeeze(0) | ||
|
|
||
| logits = model(**inputs).logits |
Contributor
Author
There was a problem hiding this comment.
The same as above, but for multiple inputs.
This PR: 1. adds support for convolutions with related tests 2. brings support for multiple inputs 3. adds the option to hide progress to SensitivityCalibrator. TICO-DCO-1.0-Signed-off-by: s.malakhov <s.malakhov@partner.samsung.com>
stamalakhov
force-pushed
the
sens_for_convs
branch
from
9e8e474 to
c499149
Compare
mhs4670go
reviewed
Mar 25, 2026
| for type in self.calibrated_types: | ||
| if isinstance(module, type): | ||
| modules_to_process[name] = module | ||
| name_of_module[module] = name |
Contributor
There was a problem hiding this comment.
This is not related with this PR but name_of_module uses nowhere.
Contributor
Author
There was a problem hiding this comment.
Ahh. You're right! Thank you! I'll remove it.
This was referenced Mar 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR:
./ccex test --include-internal -k quantization.algorithm.test_gptq.GPTQTest
Draft: #559
Related: #548
TICO-DCO-1.0-Signed-off-by: s.malakhov s.malakhov@partner.samsung.com