Skip to content

[Bugfix] Staged 2of4 example#1238

Merged
kylesayrs merged 4 commits intomainfrom
kylesayrs/fix-quantization-modifier
Mar 11, 2025
Merged

[Bugfix] Staged 2of4 example#1238
kylesayrs merged 4 commits intomainfrom
kylesayrs/fix-quantization-modifier

Conversation

@kylesayrs
Copy link
Copy Markdown
Collaborator

Purpose

  • Fix staged 2of4 example

Background

  • When [Callbacks] Remove pre_initialize_structure #1160 landed, this change introduced a bug in the recipe container which meant that the recipe was not recompiled after appending. This caused sgpt to initialize twice and gptq to never initialize, leading to a sparsity-only quantization config
  • At some point, a changed was introduced which causes previous stages to become reconstructed after recipe recompilation. This means that without resetting the session in between stages, previous stages will initialize twice.
    • In order to avoid this issue, this PR introduces session.reset() in between stages
      • This change has the consequence of creating recipe.yaml files which do not have the full recipe history. However, I believe this is acceptable for the time being, as the stage runner and this work flow will be removed in the next release.

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
@github-actions
Copy link
Copy Markdown

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

@kylesayrs kylesayrs changed the title [Bugfix] [Bugfix] Staged 2of4 example Mar 10, 2025
Copy link
Copy Markdown
Collaborator

@dsikka dsikka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just so I understand, we're removing the functionality to track past stages? Which I think landed not too long ago

@dsikka dsikka added the ready When a PR is ready for review label Mar 10, 2025
@kylesayrs
Copy link
Copy Markdown
Collaborator Author

@dsikka That functionality breaks only for this example

@kylesayrs kylesayrs enabled auto-merge (squash) March 10, 2025 23:30
@kylesayrs kylesayrs merged commit 8290679 into main Mar 11, 2025
8 checks passed
@kylesayrs kylesayrs deleted the kylesayrs/fix-quantization-modifier branch March 11, 2025 03:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready When a PR is ready for review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants