Skip to content

Try importing gptqmodel first when exporting a GPTQ model to openvino #1233

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Try importing gptqmodel first when exporting a GPTQ model to openvino #1233

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d642dde
Check for gptqmodel first when exporting a GPTQ model to openvino
notsyncing Apr 9, 2025
4f19f48
Update optimum/exporters/openvino/__main__.py
IlyasMoutawwakil May 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion optimum/exporters/openvino/__main__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@
_infer_library_from_model_name_or_path,
_OpenClipForZeroShotImageClassification,
)
from optimum.utils import is_auto_gptq_available, is_gptqmodel_available

from .utils import (
_MAX_UNCOMPRESSED_SIZE,
Expand Down Expand Up @@ -344,7 +345,12 @@ def main_export(
orig_post_init_model = GPTQQuantizer.post_init_model

def post_init_model(self, model):
from auto_gptq import exllama_set_max_input_length
if is_gptqmodel_available():
from gptqmodel import exllama_set_max_input_length
elif is_auto_gptq_available():
from auto_gptq import exllama_set_max_input_length
else:
raise ValueError("Neither ``auto_gptq nor `gptqmodel` is available, please install one of them.")

class StoreAttr(object):
pass
Expand Down