Skip to content

[TorchAO] AWQ path fails after PR2400 with `ImportError: cannot import name 'awq_uintx' from 'torchao.prototype.awq' #1919

@MingxuZh

Description

@MingxuZh

🐛 Describe the bug

AWQ path fails after PR2400 with `ImportError: cannot import name 'awq_uintx' from 'torchao.prototype.awq'

PR link: pytorch/ao#2400

log:
Traceback (most recent call last):
File "/home/zhouyi/jenkins/workspace/IPEX_XPU_LLM_TORCHAO/frameworks.ai.pytorch.gpu-models/LLM/inference/run_generation.py", line 193, in
from torchao.prototype.awq import AWQObservedLinear, awq_uintx, insert_awq_observer_
ImportError: cannot import name 'awq_uintx' from 'torchao.prototype.awq' (/home/zhouyi/jenkins/workspace/IPEX_XPU_LLM_TORCHAO/ao/torchao/prototype/awq/init.py)

cmd:
python -u run_generation.py -m meta-llama/Llama-3.2-1B --input-tokens 1024 --max-new-tokens 1024 --num-iter 8 --num-warmup 4 --batch-size 1 --load-quantize-model --num-beams 1 --inductor --use-hf-code False --use-static-cache --sub-model-name llama3.2-3b --model-save-path /mnt/local_data/huggingface/hub/AWQ/Llama-3.2-1B-AWQ-INT4_P.pt --woq --woq-type awq --quant-dtype uint4 --group-size 128 --device xpu --token-latency

script:
https://github.com/intel-innersource/frameworks.ai.pytorch.gpu-models/pull/983

Versions

torch: 2.9.0.dev20250806+xpu
torchao: 0.13.0+gitc086adee0

Metadata

Metadata

Assignees

Type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions