[Refactor] Rename offload_model and dispatch_for_sequential to set_onload_device

[nrmlthms]

Summary

• Deleted dispatch_for_sequential from
  pipelines/sequential/helpers.py — it was a thin wrapper
  around offload_model with no additional logic
• Updated SequentialPipeline to call set_onload_device
  directly from compressed_tensors.offload
• Updated tests/llmcompressor/utils/test_helpers.py to use
  set_onload_device

Closes #2483

Notes

Companion PR in compressed-tensors: vllm-project/compressed-tensors#643

[github-actions]

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refactors the model offloading mechanism within the sequential pipeline. It simplifies the codebase by removing a redundant wrapper function, dispatch_for_sequential, and instead promotes direct usage of the core set_onload_device utility. This change improves clarity and reduces unnecessary abstraction in the model dispatching logic.

Highlights

• Refactoring dispatch_for_sequential: The dispatch_for_sequential function has been removed from src/llmcompressor/pipelines/sequential/helpers.py as it was identified as a thin wrapper around offload_model without additional logic.
• Direct set_onload_device usage: The SequentialPipeline now directly calls set_onload_device from compressed_tensors.offload, streamlining the model offloading process.
• Test updates: Relevant tests in tests/llmcompressor/utils/test_helpers.py have been updated to reflect these changes, now directly utilizing set_onload_device.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request refactors the model offloading logic by removing the dispatch_for_sequential wrapper function and replacing its usage, along with calls to the now-renamed offload_model, with set_onload_device from the compressed-tensors library. The changes are applied consistently across the sequential pipeline and its tests, simplifying the codebase by removing an unnecessary abstraction.

[brian-dellabetta]

Thanks! LGTM, pending changes requested to CT PR

[kylesayrs]

Looks good! Let's make sure the CT PR lands first before merging this one.

[mergify]

The quality checks have failed. Please run make style and make quality under
the root directory to adddress the lint failures. You will need to install the
dev optional install to get the required linting packages:
https://github.com/vllm-project/llm-compressor/blob/main/CONTRIBUTING.md

[mergify]

The quality checks have failed. Please run make style and make quality under
the root directory to adddress the lint failures. You will need to install the
dev optional install to get the required linting packages:
https://github.com/vllm-project/llm-compressor/blob/main/CONTRIBUTING.md

[HDCharles]

can you remove all these one-off formatting changes?

(some of them are breaking the quality checks incidentally) but even if not, lets leave those in a separate PR if you want to make them

if you do pip install -e .[dev] in the llm compressor repo that will give you the right version of ruff that won't add 24 files worth of formatting changes