the result that imagenet model generated is not good enough, I think the diffusion config sets more steps?

[AlbertLin0]

the label picture is

the per_recon picture is

It looks like the diffusion model has not iterated enough steps, (one thousand)

[tongdaxu]

Hi @AlbertLin0, this result is pretty strange and we have never seen such a reconstruction. We believe it is a bug, can you provide the exact command line and config?

[AlbertLin0]

hi @tongdaxu,
The command line is:

--model_config=configs/imagenet_model_config.yaml
--diffusion_config=configs/diffusion_config.yaml
--task_config=configs/elic_config.yaml
--save_dir=results

the config: I just change the 'q' from 1 to 2 and others is default
and the python version is 3.9.13, I dont know whether the version of packages would cause this problem
the ffhq diffusion model has the beautiful result.

[tongdaxu]

hi @tongdaxu, The command line is:

--model_config=configs/imagenet_model_config.yaml
--diffusion_config=configs/diffusion_config.yaml
--task_config=configs/elic_config.yaml
--save_dir=results

the config: I just change the 'q' from 1 to 2 and others is default and the python version is 3.9.13, I dont know whether the version of packages would cause this problem the ffhq diffusion model has the beautiful result.

This is strange. The configs look ok to me. I am traveling with limited gpu access so I might not be able to actually run this until May 13. Channing q from 1 to 2 should not bring a huge difference. Can you please try ImageNet + hyperprior ? So I can check whether it is the problem of diffusion model or elic.

[tongdaxu]

hi @tongdaxu, The command line is:

--model_config=configs/imagenet_model_config.yaml
--diffusion_config=configs/diffusion_config.yaml
--task_config=configs/elic_config.yaml
--save_dir=results

the config: I just change the 'q' from 1 to 2 and others is default and the python version is 3.9.13, I dont know whether the version of packages would cause this problem the ffhq diffusion model has the beautiful result.

Also would you like to try a different torch version like torch 2.1 and 2.2?

the diffusion model might be buggy for torch 1.9, see huggingface/diffusers#6730 (comment)

[AlbertLin0]

hi @tongdaxu, The command line is:

--model_config=configs/imagenet_model_config.yaml
--diffusion_config=configs/diffusion_config.yaml
--task_config=configs/elic_config.yaml
--save_dir=results

the config: I just change the 'q' from 1 to 2 and others is default and the python version is 3.9.13, I dont know whether the version of packages would cause this problem the ffhq diffusion model has the beautiful result.

Also would you like to try a different torch version like torch 2.1 and 2.2?

the diffusion model might be buggy for torch 1.9, see huggingface/diffusers#6730 (comment)

I have tried torch 2.1.2 and torch 2.2.0 with imagenet config, and the results are similar as the nosiy picture above. But, the results of ffhq config are pretty good. And I'm tring to run diffusion-posterior-sampling...

[tongdaxu]

hi @tongdaxu, The command line is:

--model_config=configs/imagenet_model_config.yaml
--diffusion_config=configs/diffusion_config.yaml
--task_config=configs/elic_config.yaml
--save_dir=results

the config: I just change the 'q' from 1 to 2 and others is default and the python version is 3.9.13, I dont know whether the version of packages would cause this problem the ffhq diffusion model has the beautiful result.

Also would you like to try a different torch version like torch 2.1 and 2.2?
the diffusion model might be buggy for torch 1.9, see huggingface/diffusers#6730 (comment)

I have tried torch 2.1.2 and torch 2.2.0 with imagenet config, and the results are similar as the nosiy picture above. But, the results of ffhq config are pretty good. And I'm tring to run diffusion-posterior-sampling...

if you have time, it’s is helpful to run a sr x4 DPS and check the result. So we will know whether it’s the problem of environment or our code. And I can debug faster later.

[AlbertLin0]

hi @tongdaxu, The command line is:

--model_config=configs/imagenet_model_config.yaml
--diffusion_config=configs/diffusion_config.yaml
--task_config=configs/elic_config.yaml
--save_dir=results

the config: I just change the 'q' from 1 to 2 and others is default and the python version is 3.9.13, I dont know whether the version of packages would cause this problem the ffhq diffusion model has the beautiful result.

Also would you like to try a different torch version like torch 2.1 and 2.2?
the diffusion model might be buggy for torch 1.9, see huggingface/diffusers#6730 (comment)

I have tried torch 2.1.2 and torch 2.2.0 with imagenet config, and the results are similar as the nosiy picture above. But, the results of ffhq config are pretty good. And I'm tring to run diffusion-posterior-sampling...

if you have time, it’s is helpful to run a sr x4 DPS and check the result. So we will know whether it’s the problem of environment or our code. And I can debug faster later.

configs

I have put 'elic' operater to DPS, the args is

            "args":[
                "--model_config=configs/imagenet_model_config.yaml",
                "--diffusion_config=configs/diffusion_config.yaml",
                "--task_config=configs/elic_config.yaml"
            ]

the imagenet_model_config and diffusion_config is default, and the elic_config:

conditioning:
  method: ps
  params:
    scale: 0.3

data:
  name: ffhq
  root: ./data/samples/

measurement:
  operator:
    name: elic
    q: 2

  noise: 
    name: gaussian
    sigma: 0.05

results:

input:

label:

recon:

[tongdaxu]

hi @tongdaxu, The command line is:

--model_config=configs/imagenet_model_config.yaml
--diffusion_config=configs/diffusion_config.yaml
--task_config=configs/elic_config.yaml
--save_dir=results

the config: I just change the 'q' from 1 to 2 and others is default and the python version is 3.9.13, I dont know whether the version of packages would cause this problem the ffhq diffusion model has the beautiful result.

Also would you like to try a different torch version like torch 2.1 and 2.2?
the diffusion model might be buggy for torch 1.9, see huggingface/diffusers#6730 (comment)

I have tried torch 2.1.2 and torch 2.2.0 with imagenet config, and the results are similar as the nosiy picture above. But, the results of ffhq config are pretty good. And I'm tring to run diffusion-posterior-sampling...

if you have time, it’s is helpful to run a sr x4 DPS and check the result. So we will know whether it’s the problem of environment or our code. And I can debug faster later.

configs

I have put 'elic' operater to DPS, the args is

            "args":[
                "--model_config=configs/imagenet_model_config.yaml",
                "--diffusion_config=configs/diffusion_config.yaml",
                "--task_config=configs/elic_config.yaml"
            ]

the imagenet_model_config and diffusion_config is default, and the elic_config:

conditioning:
  method: ps
  params:
    scale: 0.3

data:
  name: ffhq
  root: ./data/samples/

measurement:
  operator:
    name: elic
    q: 2

  noise: 
    name: gaussian
    sigma: 0.05

results:

input:

label:

recon:

How is Elic on imagenet model works? Or is that already image net model and you are just testing with FFHQ?

[AlbertLin0]

hi @tongdaxu, The command line is:

--model_config=configs/imagenet_model_config.yaml
--diffusion_config=configs/diffusion_config.yaml
--task_config=configs/elic_config.yaml
--save_dir=results

the config: I just change the 'q' from 1 to 2 and others is default and the python version is 3.9.13, I dont know whether the version of packages would cause this problem the ffhq diffusion model has the beautiful result.

Also would you like to try a different torch version like torch 2.1 and 2.2?
the diffusion model might be buggy for torch 1.9, see huggingface/diffusers#6730 (comment)

I have tried torch 2.1.2 and torch 2.2.0 with imagenet config, and the results are similar as the nosiy picture above. But, the results of ffhq config are pretty good. And I'm tring to run diffusion-posterior-sampling...

if you have time, it’s is helpful to run a sr x4 DPS and check the result. So we will know whether it’s the problem of environment or our code. And I can debug faster later.

configs

I have put 'elic' operater to DPS, the args is

            "args":[
                "--model_config=configs/imagenet_model_config.yaml",
                "--diffusion_config=configs/diffusion_config.yaml",
                "--task_config=configs/elic_config.yaml"
            ]

the imagenet_model_config and diffusion_config is default, and the elic_config:

conditioning:
  method: ps
  params:
    scale: 0.3

data:
  name: ffhq
  root: ./data/samples/

measurement:
  operator:
    name: elic
    q: 2

  noise: 
    name: gaussian
    sigma: 0.05

results:

input:
label:
recon:

How is Elic on imagenet model works? Or is that already image net model and you are just testing with FFHQ?

I just want to figure out why imagenet model generate noisy picture with your code. So the test is example DPS provided.

[AuroraZengfh]

hi, @AlbertLin0, I'm also running the codebase. It looks fine for me to run through elic with imagenet. All my configs are default except for the path of pretrained model and datasets.

Label picture is:

per_recon picture is:

It looks like that your environment has something not compatible. Here's my environment and my python is 3.8.18. Hope it will help you figure out.

Package Version Editable project location

---

absl-py 2.1.0
accelerate 0.28.0
addict 2.4.0
aiosignal 1.3.1
annotated-types 0.6.0
antlr4-python3-runtime 4.9.3
anykeystore 0.2
apex 0.1
asttokens 2.4.1
attrs 23.2.0
av 12.0.0
backcall 0.2.0
bcrypt 4.1.2
beautifulsoup4 4.12.3
cachetools 5.3.3
certifi 2023.11.17
cffi 1.16.0
cfgv 3.4.0
charset-normalizer 3.3.2
click 8.1.7
cloudpickle 3.0.0
cmake 3.28.3
colossalai 0.3.6
comm 0.2.1
contexttimer 0.3.3
contourpy 1.1.1
cryptacular 1.6.2
cryptography 42.0.5
cycler 0.12.1
decorator 5.1.1
defusedxml 0.7.1
Deprecated 1.2.14
distlib 0.3.8
einops 0.7.0
executing 2.0.1
fabric 3.2.2
filelock 3.13.1
fonttools 4.50.0
frozenlist 1.4.1
fsspec 2023.12.2
ftfy 6.1.3
google 3.0.0
google-auth 2.29.0
google-auth-oauthlib 1.0.0
greenlet 3.0.3
grpcio 1.62.1
huggingface-hub 0.20.3
hupper 1.12.1
Hydra 2.5
hydra-core 1.3.2
hydra-submitit-launcher 1.2.0
identify 2.5.35
idna 3.6
importlib-metadata 7.0.1
importlib-resources 6.1.1
invoke 2.2.0
ipython 8.12.3
ipywidgets 8.1.1
jedi 0.19.1
Jinja2 3.1.3
jsonschema 4.21.1
jsonschema-specifications 2023.12.1
jupyterlab-widgets 3.0.9
kiwisolver 1.4.5
lit 18.1.1
Markdown 3.6
markdown-it-py 3.0.0
MarkupSafe 2.1.5
matplotlib 3.7.5
matplotlib-inline 0.1.6
mdurl 0.1.2
mmengine 0.10.3
mpmath 1.3.0
msgpack 1.0.8
mypy-extensions 1.0.0
networkx 3.1
ninja 1.11.1.1
nodeenv 1.8.0
numpy 1.24.4
nvidia-cublas-cu11 11.10.3.66
nvidia-cublas-cu12 12.1.3.1
nvidia-cuda-cupti-cu11 11.7.101
nvidia-cuda-cupti-cu12 12.1.105
nvidia-cuda-nvrtc-cu11 11.7.99
nvidia-cuda-nvrtc-cu12 12.1.105
nvidia-cuda-runtime-cu11 11.7.99
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu11 8.5.0.96
nvidia-cudnn-cu12 8.9.2.26
nvidia-cufft-cu11 10.9.0.58
nvidia-cufft-cu12 11.0.2.54
nvidia-curand-cu11 10.2.10.91
nvidia-curand-cu12 10.3.2.106
nvidia-cusolver-cu11 11.4.0.1
nvidia-cusolver-cu12 11.4.5.107
nvidia-cusparse-cu11 11.7.4.91
nvidia-cusparse-cu12 12.1.0.106
nvidia-nccl-cu11 2.14.3
nvidia-nccl-cu12 2.19.3
nvidia-nvjitlink-cu12 12.4.99
nvidia-nvtx-cu11 11.7.91
nvidia-nvtx-cu12 12.1.105
oauthlib 3.2.2
omegaconf 2.3.0
opencv-python 4.9.0.80
opt-einsum 3.3.0
packaging 23.2
paramiko 3.4.0
parso 0.8.3
PasteDeploy 3.1.0
pbkdf2 1.3
pexpect 4.9.0
pickleshare 0.7.5
pillow 10.2.0
pip 23.3.1
pkgutil_resolve_name 1.3.10
plaster 1.1.2
plaster-pastedeploy 1.0.1
platformdirs 4.2.0
pre-commit 3.5.0
prompt-toolkit 3.0.43
protobuf 5.26.0
psutil 5.9.8
ptyprocess 0.7.0
pure-eval 0.2.2
pyasn1 0.5.1
pyasn1-modules 0.3.0
pycparser 2.21
pydantic 2.6.4
pydantic_core 2.16.3
Pygments 2.17.2
PyNaCl 1.5.0
pyparsing 3.1.2
pyramid 2.0.2
pyramid-mailer 0.15.1
pyre-extensions 0.0.23
python-dateutil 2.9.0.post0
python3-openid 3.2.0
PyYAML 6.0.1
ray 2.9.3
referencing 0.34.0
regex 2023.12.25
repoze.sendmail 4.4.1
requests 2.31.0
requests-oauthlib 2.0.0
rich 13.7.1
rpds-py 0.18.0
rsa 4.9
safetensors 0.4.2
sentencepiece 0.2.0
setuptools 68.2.2
six 1.16.0
soupsieve 2.5
SQLAlchemy 2.0.28
stack-data 0.6.3
submitit 1.5.1
sympy 1.12
tensorboard 2.14.0
tensorboard-data-server 0.7.2
termcolor 2.4.0
timm 0.4.12
tokenizers 0.15.1
tomli 2.0.1
torch 2.0.0+cu118
torchvision 0.15.1+cu118
tqdm 4.66.1
traitlets 5.14.1
transaction 4.0
transformers 4.37.1
translationstring 1.4
triton 2.0.0
typing_extensions 4.9.0
typing-inspect 0.9.0
urllib3 2.1.0
velruse 1.1.1
venusian 3.1.0
virtualenv 20.25.1
wcwidth 0.2.13
WebOb 1.8.7
Werkzeug 3.0.1
wheel 0.41.2
widgetsnbextension 4.0.9
wrapt 1.16.0
WTForms 3.1.2
wtforms-recaptcha 0.3.2
xformers 0.0.17
yapf 0.40.2
zipp 3.18.1
zope.deprecation 5.0
zope.interface 6.2
zope.sqlalchemy 3.1

[tongdaxu]

Hi @AlbertLin0, I have asked my intern @AuroraZengfh to run the ELIC Imagenet from scratch, and the result is fine. I have asked him to post the result and his environment. Have you solved this issue? Aligning to his environment and try again might be helpful.