Getting RuntimeError: expected mat1 and mat2 to have the same dtype, but got: struct c10::Half != struct c10::BFloat16 Error #15
Description
Hi,
I'm getting the following error when running a basic workflow. This also happened previously with the custom node you forked from, so perhaps there is some code that needs to be pulled? I did a git clone of your repository and installed the requirements.
Model: fast, Res: 1024x1024, Steps: 16, CFG: 0.0, Seed: 0
Sequence lengths - CLIP-L: 77, OpenCLIP: 77, T5: 128, Llama: 128
Ensuring pipe on: cuda:0 (Offload NOT enabled)
Executing pipeline inference...
Using per-encoder prompts:
CLIP-L (77 tokens): ...
OpenCLIP (77 tokens): ...
T5 (128 tokens): ...
Llama (128 tokens): ...
!!! ERROR during execution: expected mat1 and mat2 to have the same dtype, but got: struct c10::Half != struct c10::BFloat16
Traceback (most recent call last):
File "D:\AI\ComfyUI_windows_portable_main_hidreamtest\ComfyUI\custom_nodes\ComfyUI-HiDream-Sampler\hidreamsampler.py", line 1022, in generate
output_images = pipe(
^^^^^
File "D:\AI\ComfyUI_windows_portable_main_hidreamtest\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "D:\AI\ComfyUI_windows_portable_main_hidreamtest\ComfyUI\custom_nodes\ComfyUI-HiDream-Sampler\hi_diffusers\pipelines\hidream_image\pipeline_hidream_image.py", line 637, in call
) = self.encode_prompt(
^^^^^^^^^^^^^^^^^^^
File "D:\AI\ComfyUI_windows_portable_main_hidreamtest\ComfyUI\custom_nodes\ComfyUI-HiDream-Sampler\hi_diffusers\pipelines\hidream_image\pipeline_hidream_image.py", line 322, in encode_prompt
prompt_embeds, pooled_prompt_embeds = self._encode_prompt(
^^^^^^^^^^^^^^^^^^^^
File "D:\AI\ComfyUI_windows_portable_main_hidreamtest\ComfyUI\custom_nodes\ComfyUI-HiDream-Sampler\hi_diffusers\pipelines\hidream_image\pipeline_hidream_image.py", line 435, in _encode_prompt
pooled_prompt_embeds_1 = self._get_clip_prompt_embeds(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\AI\ComfyUI_windows_portable_main_hidreamtest\ComfyUI\custom_nodes\ComfyUI-HiDream-Sampler\hi_diffusers\pipelines\hidream_image\pipeline_hidream_image.py", line 219, in _get_clip_prompt_embeds
prompt_embeds = text_encoder(text_input_ids.to(device), output_hidden_states=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\AI\ComfyUI_windows_portable_main_hidreamtest\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1751, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\AI\ComfyUI_windows_portable_main_hidreamtest\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1762, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\AI\ComfyUI_windows_portable_main_hidreamtest\python_embeded\Lib\site-packages\transformers\models\clip\modeling_clip.py", line 1490, in forward
text_embeds = self.text_projection(pooled_output)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\AI\ComfyUI_windows_portable_main_hidreamtest\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1751, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\AI\ComfyUI_windows_portable_main_hidreamtest\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1762, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\AI\ComfyUI_windows_portable_main_hidreamtest\python_embeded\Lib\site-packages\torch\nn\modules\linear.py", line 125, in forward
return F.linear(input, self.weight, self.bias)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: expected mat1 and mat2 to have the same dtype, but got: struct c10::Half != struct c10::BFloat16
Prompt executed in 149.01 seconds