RuntimeError: Failed to import diffusers.pipelines.cogvideo.pipeline_cogvideox because of the following error (look up to see its traceback):

[KaifAhmad1]

Describe the bug

Getting this issue when running the cogvideox model on Google Tesla T4 GPU text to video generation use case

!pip install -q git+https://github.com/huggingface/diffusers.git
!pip install -q torch transformers bitsandbytes torchao accelerate psutil

import torch
import time
import psutil
from accelerate import Accelerator
from diffusers import CogVideoXPipeline
from diffusers.utils import export_to_video

Reproduction

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
File /usr/local/lib/python3.11/dist-packages/diffusers/utils/import_utils.py:839, in _LazyModule._get_module(self, module_name)
    838 try:
--> 839     return importlib.import_module("." + module_name, self.__name__)
    840 except Exception as e:

File /usr/lib/python3.11/importlib/__init__.py:126, in import_module(name, package)
    125         level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1204, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1176, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1147, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:690, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:940, in exec_module(self, module)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

File /usr/local/lib/python3.11/dist-packages/diffusers/loaders/lora_pipeline.py:34
     21 from ..utils import (
     22     USE_PEFT_BACKEND,
     23     deprecate,
   (...)
     32     logging,
     33 )
---> 34 from .lora_base import (  # noqa
     35     LORA_WEIGHT_NAME,
     36     LORA_WEIGHT_NAME_SAFE,
     37     LoraBaseMixin,
     38     _fetch_state_dict,
     39     _load_lora_into_text_encoder,
     40 )
     41 from .lora_conversion_utils import (
     42     _convert_bfl_flux_control_lora_to_diffusers,
     43     _convert_hunyuan_video_lora_to_diffusers,
   (...)
     50     _maybe_map_sgm_blocks_to_diffusers,
     51 )

File /usr/local/lib/python3.11/dist-packages/diffusers/loaders/lora_base.py:27
     25 from huggingface_hub.constants import HF_HUB_OFFLINE
---> 27 from ..models.modeling_utils import ModelMixin, load_state_dict
     28 from ..utils import (
     29     USE_PEFT_BACKEND,
     30     _get_model_file,
   (...)
     46     set_weights_and_activate_adapters,
     47 )

File /usr/local/lib/python3.11/dist-packages/diffusers/models/modeling_utils.py:40
     39 from .. import __version__
---> 40 from ..quantizers import DiffusersAutoQuantizer, DiffusersQuantizer
     41 from ..quantizers.quantization_config import QuantizationMethod

File /usr/local/lib/python3.11/dist-packages/diffusers/quantizers/__init__.py:15
      1 # Copyright 2024 The HuggingFace Team. All rights reserved.
      2 #
      3 # Licensed under the Apache License, Version 2.0 (the "License");
   (...)
     12 # See the License for the specific language governing permissions and
     13 # limitations under the License.
---> 15 from .auto import DiffusersAutoQuantizer
     16 from .base import DiffusersQuantizer

File /usr/local/lib/python3.11/dist-packages/diffusers/quantizers/auto.py:22
     20 from typing import Dict, Optional, Union
---> 22 from .bitsandbytes import BnB4BitDiffusersQuantizer, BnB8BitDiffusersQuantizer
     23 from .gguf import GGUFQuantizer

File /usr/local/lib/python3.11/dist-packages/diffusers/quantizers/bitsandbytes/__init__.py:1
----> 1 from .bnb_quantizer import BnB4BitDiffusersQuantizer, BnB8BitDiffusersQuantizer
      2 from .utils import dequantize_and_replace, dequantize_bnb_weight, replace_with_bnb_linear

File /usr/local/lib/python3.11/dist-packages/diffusers/quantizers/bitsandbytes/bnb_quantizer.py:22
     21 from ...utils import get_module_from_name
---> 22 from ..base import DiffusersQuantizer
     25 if TYPE_CHECKING:

File /usr/local/lib/python3.11/dist-packages/diffusers/quantizers/base.py:24
     23 from ..utils import is_torch_available
---> 24 from .quantization_config import QuantizationConfigMixin
     27 if TYPE_CHECKING:

File /usr/local/lib/python3.11/dist-packages/diffusers/quantizers/quantization_config.py:52
     51 if is_torchao_available():
---> 52     from torchao.quantization.quant_primitives import MappingType
     54     class TorchAoJSONEncoder(json.JSONEncoder):

File /usr/local/lib/python3.11/dist-packages/torchao/__init__.py:41
     39     logging.debug(f"Skipping import of cpp extensions: {e}")
---> 41 from torchao.quantization import (
     42     autoquant,
     43     quantize_,
     44 )
     46 from . import dtypes, optim, testing

File /usr/local/lib/python3.11/dist-packages/torchao/quantization/__init__.py:1
----> 1 from torchao.kernel import (
      2     int_scaled_matmul,
      3     safe_int_mm,
      4 )
      6 from .autoquant import (
      7     ALL_AUTOQUANT_CLASS_LIST,
      8     DEFAULT_AUTOQUANT_CLASS_LIST,
   (...)
     14     autoquant,
     15 )

File /usr/local/lib/python3.11/dist-packages/torchao/kernel/__init__.py:1
----> 1 from torchao.kernel.bsr_triton_ops import bsr_dense_addmm
      2 from torchao.kernel.intmm import int_scaled_matmul, safe_int_mm

File /usr/local/lib/python3.11/dist-packages/torchao/kernel/bsr_triton_ops.py:28
     21 from torch.sparse._triton_ops import (
     22     broadcast_batch_dims,
     23     launch_kernel,
   (...)
     26     tile_to_blocksize,
     27 )
---> 28 from torch.sparse._triton_ops_meta import get_meta, minimize, update
     29 from torch.utils._triton import has_triton

ModuleNotFoundError: No module named 'torch.sparse._triton_ops_meta'

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
File /usr/local/lib/python3.11/dist-packages/diffusers/utils/import_utils.py:839, in _LazyModule._get_module(self, module_name)
    838 try:
--> 839     return importlib.import_module("." + module_name, self.__name__)
    840 except Exception as e:

File /usr/lib/python3.11/importlib/__init__.py:126, in import_module(name, package)
    125         level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1204, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1176, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1147, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:690, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:940, in exec_module(self, module)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

File /usr/local/lib/python3.11/dist-packages/diffusers/pipelines/cogvideo/pipeline_cogvideox.py:24
     23 from ...callbacks import MultiPipelineCallbacks, PipelineCallback
---> 24 from ...loaders import CogVideoXLoraLoaderMixin
     25 from ...models import AutoencoderKLCogVideoX, CogVideoXTransformer3DModel

File <frozen importlib._bootstrap>:1229, in _handle_fromlist(module, fromlist, import_, recursive)

File /usr/local/lib/python3.11/dist-packages/diffusers/utils/import_utils.py:829, in _LazyModule.__getattr__(self, name)
    828 elif name in self._class_to_module.keys():
--> 829     module = self._get_module(self._class_to_module[name])
    830     value = getattr(module, name)

File /usr/local/lib/python3.11/dist-packages/diffusers/utils/import_utils.py:841, in _LazyModule._get_module(self, module_name)
    840 except Exception as e:
--> 841     raise RuntimeError(
    842         f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its"
    843         f" traceback):\n{e}"
    844     ) from e

RuntimeError: Failed to import diffusers.loaders.lora_pipeline because of the following error (look up to see its traceback):
No module named 'torch.sparse._triton_ops_meta'

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
Cell In[3], line 5
      3 import psutil
      4 from accelerate import Accelerator
----> 5 from diffusers import CogVideoXPipeline
      6 from diffusers.utils import export_to_video

File <frozen importlib._bootstrap>:1229, in _handle_fromlist(module, fromlist, import_, recursive)

File /usr/local/lib/python3.11/dist-packages/diffusers/utils/import_utils.py:830, in _LazyModule.__getattr__(self, name)
    828 elif name in self._class_to_module.keys():
    829     module = self._get_module(self._class_to_module[name])
--> 830     value = getattr(module, name)
    831 else:
    832     raise AttributeError(f"module {self.__name__} has no attribute {name}")

File /usr/local/lib/python3.11/dist-packages/diffusers/utils/import_utils.py:830, in _LazyModule.__getattr__(self, name)
    828 elif name in self._class_to_module.keys():
    829     module = self._get_module(self._class_to_module[name])
--> 830     value = getattr(module, name)
    831 else:
    832     raise AttributeError(f"module {self.__name__} has no attribute {name}")

File /usr/local/lib/python3.11/dist-packages/diffusers/utils/import_utils.py:829, in _LazyModule.__getattr__(self, name)
    827     value = self._get_module(name)
    828 elif name in self._class_to_module.keys():
--> 829     module = self._get_module(self._class_to_module[name])
    830     value = getattr(module, name)
    831 else:

File /usr/local/lib/python3.11/dist-packages/diffusers/utils/import_utils.py:841, in _LazyModule._get_module(self, module_name)
    839     return importlib.import_module("." + module_name, self.__name__)
    840 except Exception as e:
--> 841     raise RuntimeError(
    842         f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its"
    843         f" traceback):\n{e}"
    844     ) from e

RuntimeError: Failed to import diffusers.pipelines.cogvideo.pipeline_cogvideox because of the following error (look up to see its traceback):
Failed to import diffusers.loaders.lora_pipeline because of the following error (look up to see its traceback):
No module named 'torch.sparse._triton_ops_meta'

Logs

System Info

Windows 11 x64
python 3.11.9
cuda 12.1

Who can help?

@DN6 @a-r-r-o-w

[asomoza]

Hi, this error sounds similar to this one.

I tested your code with windows and it loads without a problem, if you install torchao it will suggest you that you have to install triton because some kernels will not work.

Basically you have a broken environment, you will need to clean it and install only what you need, if you plan to use torchao and triton, those aren't easy to install and run with windows and it's out of scope of what we can help you with.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[DN6]

@KaifAhmad1 Closing this for now. Feel free to reopen if the issue is not resolved with a fresh environment.