CUDA error: invalid device ordinal

[qxu99gh]

I got "RuntimeError: CUDA error: invalid device ordinal" at \1Prompt1Story\app.py", line 26, in main_gradio
pipe, _ = utils.load_pipe_from_path(model_path, "cuda:1", torch.float16 if precision == "fp16" else torch.float32, precision).

I tried to set cuda:0 in app.py, but got "RuntimeError: expected mat1 and mat2 to have the same dtype, but got: float != struct c10::Half".

Anyway to fix these errors? Thanks

[byliutao]

you should also change cuda:1 to cuda:0 in unet_controller = load_unet_controller(pipe, "cuda:1") (app.py line39)

[qxu99gh]

I updated both lines to "cuda:0", but got "RuntimeError: expected mat1 and mat2 to have the same dtype, but got: float != struct c10::Half".

[byliutao]

I have tested app.py with cuda:0, it's running fine. This error looks like a precision mismatch, potentially caused by mixing use of FP16 and FP32. Could you check the specific line where the error occurs to see if all the variables are of the same precision？

[Byunggwan-Son]

I encountered the same error when running main.py. It looks like the text encoders in lines 210-211 of utils.py have mismatched data types—text_encoder is torch.float16, while text_encoder_2 is torch.float32. The error was fixed by adding text_encoder_2.to(dtype=torch_dtype). I wonder if it is a problem with the transformers library.

[byliutao]

That's a bit wild, because in utils.py(line 210 to 211) I have set the dtype of the two text encoders to the same one. I am using transformers==4.46.3. I will update the specific version in the readme.

[byliutao]

For those encountering this issue, I recommend adding pipe.to(torch_dtype) after line 223 in utils.py to ensure all modules are loaded with a uniform data type.