Seems 1.5.6 nightly was throwing this error anytime it hit the Ksampler.
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat2 in method wrapper_CUDA_mm)
I had animatediff in the pipeline as well, not sure if that matters but I didn't test thoroughly.
Rolling back to 1.5.5 fixed the issue
This was with portable comfy and manager versions below
ComfyUI: v0.3.49
(2025-08-05)
Manager: V3.32.5