cannot import name 'default_decompositions' from 'torch.export' (Pytorch & Executorch issues) #7434
Description
🐛 Describe the bug
I am trying to serve Llama 3.2 1B spinquant on my iPhone using executorch, but I am encountering errors. (macOS Sequoia 15.1, macbook Air m1 ram16gb, anaconda environment user)
ImportError: cannot import name 'default_decompositions' from 'torch.export' (/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/torch/export/__init__.py)
Let me explain the situation in detail.
Setup ExecuTorch
In this section, we will need to set up the ExecuTorch repo first with Conda environment management. Make sure you have Conda available in your system (or follow the instructions to install it here). The commands below are running on Linux (CentOS).
Create a Conda environment
conda create -n et_xnnpack python=3.10.0
conda activate et_xnnpack
Checkout ExecuTorch repo and sync submodules
git clone https://github.com/pytorch/executorch.git
cd executorch
git submodule sync
git submodule update --init
Install dependencies
./install_requirements.sh
I was able to proceed without major issues up to this point. The problem arose in the next part, where I needed to input the paths corresponding to the model.
For Llama 3.2 1B and 3B SpinQuant models
Meta has released prequantized INT4 SpinQuant Llama 3.2 models that ExecuTorch supports on the XNNPACK backend.
Export Llama model and generate .pte file as below:
python -m examples.models.llama.export_llama --model "llama3_2" --checkpoint <path-to-your-checkpoint.pth> --params <path-to-your-params.json> -kv --use_sdpa_with_kv_cache -X -d fp32 --xnnpack-extended-ops --preq_mode 8da4w_output_8da8w --preq_group_size 32 --max_seq_length 2048 --preq_embedding_quantize 8,0 --use_spin_quant native --metadata '{"get_bos_id":128000, "get_eos_ids":[128009, 128001]}' --output_name "llama3_2_spinquant.pte"
Errors arose in these codes.
(et_xnnpack) questionone@curious-MacBookAir executorch % python -m examples.models.llama.export_llama \
--model "llama3_2" \
--checkpoint "/Users/questionone/.llama/checkpoints/Llama3.2-1B-Instruct-int4-spinquant-eo8/consolidated.00.pth" \
--params "/Users/questionone/.llama/checkpoints/Llama3.2-1B-Instruct-int4-spinquant-eo8/params.json" \
-kv \
--use_sdpa_with_kv_cache \
-X \
-d fp32 \
--xnnpack-extended-ops \
--preq_mode 8da4w_output_8da8w \
--preq_group_size 32 \
--max_seq_length 2048 \
--preq_embedding_quantize 8,0 \
--use_spin_quant native \
--metadata '{"get_bos_id":128000, "get_eos_ids":[128009, 128001]}' \
--output_name "llama3_2_spinquant.pte"
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/Users/questionone/executorch/examples/models/llama/export_llama.py", line 14, in <module>
from .export_llama_lib import build_args_parser, export_llama
File "/Users/questionone/executorch/examples/models/llama/export_llama_lib.py", line 25, in <module>
from executorch.devtools.etrecord import generate_etrecord
File "/Users/questionone/executorch/devtools/__init__.py", line 7, in <module>
import executorch.devtools.inspector as inspector
File "/Users/questionone/executorch/devtools/inspector/__init__.py", line 9, in <module>
from executorch.devtools.inspector._inspector import (
File "/Users/questionone/executorch/devtools/inspector/_inspector.py", line 31, in <module>
import executorch.devtools.etdump.schema_flatcc as flatcc
File "/Users/questionone/executorch/devtools/etdump/schema_flatcc.py", line 18, in <module>
from executorch.exir.scalar_type import ScalarType
File "/Users/questionone/executorch/exir/__init__.py", line 9, in <module>
from executorch.exir.capture import (
File "/Users/questionone/executorch/exir/capture/__init__.py", line 9, in <module>
from executorch.exir.capture._capture import (
File "/Users/questionone/executorch/exir/capture/_capture.py", line 15, in <module>
from executorch.exir.capture._config import CaptureConfig
File "/Users/questionone/executorch/exir/capture/_config.py", line 15, in <module>
from executorch.exir.passes import MemoryPlanningPass, ToOutVarPass
File "/Users/questionone/executorch/exir/passes/__init__.py", line 19, in <module>
from executorch.exir import control_flow, memory, memory_planning
File "/Users/questionone/executorch/exir/control_flow.py", line 58, in <module>
from executorch.exir.tracer import (
File "/Users/questionone/executorch/exir/tracer.py", line 50, in <module>
from torch.export import default_decompositions
ImportError: cannot import name 'default_decompositions' from 'torch.export' (/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/torch/export/__init__.py)
I would like to know why these errors are occurring. Executorch and PyTorch are already installed in the environment called et_xnnpack.
Even after resetting the path with the command export PYTHONPATH=/Users/questionone/executorch, the errors persist.
Versions
(et_xnnpack) questionone@curious-MacBookAir executorch % conda list -n et_xnnpack
packages in environment at /opt/anaconda3/envs/et_xnnpack:
(et_xnnpack) questionone@curious-MacBookAir executorch % conda list -n et_xnnpack
packages in environment at /opt/anaconda3/envs/et_xnnpack:
Name Version Build Channel
annotated-types 0.7.0 pypi_0 pypi
anyio 4.7.0 pypi_0 pypi
attrs 24.3.0 pypi_0 pypi
blas 1.0 openblas
blobfile 3.0.0 pypi_0 pypi
bzip2 1.0.8 h80987f9_6
ca-certificates 2024.11.26 hca03da5_0
certifi 2024.12.14 pypi_0 pypi
charset-normalizer 3.4.0 pypi_0 pypi
click 8.1.8 pypi_0 pypi
cmake 3.31.2 pypi_0 pypi
distro 1.9.0 pypi_0 pypi
exceptiongroup 1.2.2 pypi_0 pypi
execnet 2.1.1 pypi_0 pypi
executorch 0.5.0a0+82763a9 pypi_0 pypi
expecttest 0.3.0 pypi_0 pypi
filelock 3.16.1 pypi_0 pypi
fire 0.7.0 pypi_0 pypi
flatbuffers 24.3.25 pypi_0 pypi
fsspec 2024.12.0 pypi_0 pypi
gmp 6.2.1 hc377ac9_3
gmpy2 2.1.2 py310h8c48613_0
h11 0.14.0 pypi_0 pypi
httpcore 1.0.7 pypi_0 pypi
httpx 0.28.1 pypi_0 pypi
huggingface-hub 0.27.0 pypi_0 pypi
hypothesis 6.123.0 pypi_0 pypi
idna 3.10 pypi_0 pypi
iniconfig 2.0.0 pypi_0 pypi
jinja2 3.1.5 pypi_0 pypi
libabseil 20240116.2 cxx17_h313beb8_0
libcxx 14.0.6 h848a8c0_0
libffi 3.4.4 hca03da5_1
libgfortran 5.0.0 11_3_0_hca03da5_28
libgfortran5 11.3.0 h009349e_28
libopenblas 0.3.21 h269037a_0
libprotobuf 4.25.3 h514c7bf_0
llama-models 0.0.63 pypi_0 pypi
llama-stack 0.0.63 pypi_0 pypi
llama-stack-client 0.0.63 pypi_0 pypi
llvm-openmp 14.0.6 hc6e5704_0
lxml 5.3.0 pypi_0 pypi
markdown-it-py 3.0.0 pypi_0 pypi
markupsafe 3.0.2 pypi_0 pypi
mdurl 0.1.2 pypi_0 pypi
mpc 1.1.0 h8c48613_1
mpfr 4.0.2 h695f6f0_1
mpmath 1.3.0 py310hca03da5_0
ncurses 6.4 h313beb8_0
networkx 3.4.2 pypi_0 pypi
numpy 1.21.3 pypi_0 pypi
numpy-base 1.26.4 py310ha9811e2_0
openssl 3.0.15 h80987f9_0
packaging 24.2 pypi_0 pypi
pandas 2.0.3 pypi_0 pypi
parameterized 0.9.0 pypi_0 pypi
pillow 11.0.0 pypi_0 pypi
pip 24.2 py310hca03da5_0
pluggy 1.5.0 pypi_0 pypi
prompt-toolkit 3.0.48 pypi_0 pypi
pyaml 24.12.1 pypi_0 pypi
pycryptodomex 3.21.0 pypi_0 pypi
pydantic 2.10.4 pypi_0 pypi
pydantic-core 2.27.2 pypi_0 pypi
pygments 2.18.0 pypi_0 pypi
pytest 8.3.4 pypi_0 pypi
pytest-xdist 3.6.1 pypi_0 pypi
python 3.10.16 hb885b13_1
python-dateutil 2.9.0.post0 pypi_0 pypi
python-dotenv 1.0.1 pypi_0 pypi
pytorch 2.3.1 gpu_mps_py310h7b7e308_100
pytz 2024.2 pypi_0 pypi
pyyaml 6.0.2 pypi_0 pypi
readline 8.2 h1a28f6b_0
regex 2024.11.6 pypi_0 pypi
requests 2.32.3 pypi_0 pypi
rich 13.9.4 pypi_0 pypi
ruamel-yaml 0.18.6 pypi_0 pypi
ruamel-yaml-clib 0.2.12 pypi_0 pypi
safetensors 0.4.5 pypi_0 pypi
setuptools 75.1.0 py310hca03da5_0
six 1.17.0 pypi_0 pypi
sleef 3.5.1 h80987f9_2
sniffio 1.3.1 pypi_0 pypi
sortedcontainers 2.4.0 pypi_0 pypi
sqlite 3.45.3 h80987f9_0
sympy 1.13.1 pypi_0 pypi
tabulate 0.9.0 pypi_0 pypi
termcolor 2.5.0 pypi_0 pypi
tiktoken 0.8.0 pypi_0 pypi
timm 1.0.7 pypi_0 pypi
tk 8.6.14 h6ba3021_0
tokenizers 0.20.3 pypi_0 pypi
tomli 2.2.1 pypi_0 pypi
torch 2.6.0.dev20241218 pypi_0 pypi
torchaudio 2.6.0.dev20241218 pypi_0 pypi
torchsr 1.0.4 pypi_0 pypi
torchvision 0.22.0.dev20241218 pypi_0 pypi
tqdm 4.67.1 pypi_0 pypi
transformers 4.46.1 pypi_0 pypi
typing_extensions 4.12.2 py310hca03da5_0
tzdata 2024.2 pypi_0 pypi
urllib3 2.3.0 pypi_0 pypi
wcwidth 0.2.13 pypi_0 pypi
wheel 0.44.0 py310hca03da5_0
xz 5.4.6 h80987f9_1
zlib 1.2.13 h18a0788_1
zstd 1.5.5.1 pypi_0 pypi