Error when using my own image

[munib94]

Hi,

When I execute the following command on a blurry image:

python main.py --ni --config imagenet_256_cc.yml --path_y ../../../artifacts_dataset/small_dataset/val/0145.jpg --eta 0.85 --deg "deblur_gauss" --deg_scale 2.0 --sigma_y 0.1 -i demo

I get the following error:

INFO - main.py - 2024-05-02 02:24:38,466 - Using device: cuda
2.21GB [02:55, 12.6MB/s]                                                                                                                                                                                                                                                                                   
ERROR - main.py - 2024-05-02 02:27:39,674 - Traceback (most recent call last):
  File "main.py", line 166, in main
    runner.sample(args.simplified)
  File "/home/user/MSU/Research/DDNM/guided_diffusion/diffusion.py", line 161, in sample
    model.load_state_dict(torch.load(ckpt, map_location=self.device))
  File "/home/user/.pyenv/versions/ddnm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 2189, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for UNetModel:
        Missing key(s) in state_dict: "label_emb.weight". 

Can you tell me what I am doing wrong here? I am using the pretrained model for restoring general images.

[wyhuai]

Hi, we didn't implement the code for noisy deblur.

[munib94]

Ok, in that case, I tried the following:

python main.py --ni --simplified --config imagenet_256_cc.yml --path_y ../artifacts_dataset/0050_siteA10_single_camera_ground_deblur_75/input_images/image_000008.jpg --eta 0.85 --deg "sr_averagepooling" --deg_scale 1.0 --sigma_y 0.1 -i diy

and got this error:

INFO - main.py - 2024-05-03 17:51:48,548 - Using device: cuda
Run Simplified DDNM, without SVD. 100 sampling steps. travel_length = 1, travel_repeat = 1. Task: sr_averagepooling.
building dataset from exp/imagenet_val_1k.txt
read meta done
Dataset has size 8
/home/user/.pyenv/versions/ddnm/lib/python3.8/site-packages/torch/utils/data/dataloader.py:558: UserWarning: This DataLoader will create 32 worker processes in total. Our suggested max number of worker in current system is 16, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
  warnings.warn(_create_warning_msg(
args.deg: sr_averagepooling
Start from 0
  0%|                                                                                                                                                                                                                                                              | 0/100 [00:00<?, ?it/s]
  0%|                                                                                                                                                                                                                                                                | 0/8 [00:00<?, ?it/s]
ERROR - main.py - 2024-05-03 17:51:55,236 - Traceback (most recent call last):
  File "main.py", line 166, in main
    runner.sample(args.simplified)
  File "/home/user/MSU/Research/DDNM/guided_diffusion/diffusion.py", line 199, in sample
    self.simplified_ddnm_plus(model, cls_fn)
  File "/home/user/MSU/Research/DDNM/guided_diffusion/diffusion.py", line 358, in simplified_ddnm_plus
    et = model(xt, t)
  File "/home/user/.pyenv/versions/ddnm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/user/.pyenv/versions/ddnm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/user/.pyenv/versions/ddnm/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 183, in forward
    return self.module(*inputs[0], **module_kwargs[0])
  File "/home/user/.pyenv/versions/ddnm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/user/.pyenv/versions/ddnm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/user/MSU/Research/DDNM/guided_diffusion/unet.py", line 644, in forward
    assert (y is not None) == (
AssertionError: must specify y if and only if the model is class-conditional

Can you tell me what I am missing here?

[wyhuai]

Did you use the class-conditional model without providing conditions?

[munib94]

I have both 256x256_classifier.pt and 256x256_diffusion_uncond.pt in the path DDNM/exp/logs/imagenet. If I remove 256x256_classifier.pt, I get the error,

INFO - main.py - 2024-05-05 22:06:05,422 - Using device: cuda
ERROR - main.py - 2024-05-05 22:06:10,654 - Traceback (most recent call last):
  File "main.py", line 166, in main
    runner.sample(args.simplified)
  File "/home/munib/MSU/Research/DDNM/guided_diffusion/diffusion.py", line 172, in sample
    'https://openaipublic.blob.core.windows.net/diffusion/jul-2021/%dx%d_classifier.pt' % image_size, ckpt)
TypeError: not enough arguments for format string

I am not sure which conditions you are referring to? Can you direct me to where in the code I can find the conditions it is expecting?

[wyhuai]

Can you run evaluation.sh successfully?

[munib94]

Yes, it runs successfully. I also think there is a bug when I run the same command without the --simplified option. The path to the input images defaults to DDNM/exp/datasets/imagenet/imagenet instead of the path that I provide.

[wyhuai]

Errors may occur when using personal configurations. I recommend trying it step by step based on the reference evaluation.sh