Unable to compute metrics for model output/lego

[yangqinhui0423]

Hello，everyone！
I trained it on the NeRF-synthetic dataset, and it worked, and it rendered. But when I run python metrics.py -m output/lego, I get an error saying Unable to compute metrics for model output/lego.

Has anyone encountered a similar problem and knew how to solve it?
PS：I was able to compute metrics on the Mip-NeRF360 dataset after colmap.

[RetroFuturism7]

same issue.

[curious-energy]

I made a simple change, please see #632.

[curious-energy]

I made a simple change, please see #632.

and another reason is VRAM OOM when the progress bar is interrupted.

[LakeofRain]

I made a simple change, please see #632.

I have solved the problem of 'Unable to compute metrics for model ' by 'pip install torchmetrics', although don't know why, but it does work well. Now I can obtain the metrics.

[leblond14u]

Same problem here metrics.py does not work. Tried to install torchmetrics but:

• No improvements with pip install
• And the conda install of torchmetrics is messing with my conda environment

This is what I have after a full metrics evaluation :

All of the metrics are NaN ...

Sometimes it also crashes :

Traceback (most recent call last):
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/metrics.py", line 150, in <module>
    evaluate(args.model_paths)
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/metrics.py", line 116, in evaluate
    lpipss.append(lpips(renders[idx], gts[idx], net_type='vgg'))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/lpipsPyTorch/__init__.py", line 20, in lpips
    criterion = LPIPS(net_type, version).to(device)
                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/lpipsPyTorch/modules/lpips.py", line 28, in __init__
    self.lin.load_state_dict(get_state_dict(net_type, version))
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/lpipsPyTorch/modules/utils.py", line 17, in get_state_dict
    old_state_dict = torch.hub.load_state_dict_from_url(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/anaconda3/envs/GS/lib/python3.11/site-packages/torch/hub.py", line 770, in load_state_dict_from_url
    return torch.load(cached_file, map_location=map_location, weights_only=weights_only)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/anaconda3/envs/GS/lib/python3.11/site-packages/torch/serialization.py", line 1028, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/anaconda3/envs/GS/lib/python3.11/site-packages/torch/serialization.py", line 1264, in _legacy_load
    typed_storage._untyped_storage._set_from_file(
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

[shenqianxiang]

我做了一个简单的更改，请参阅#632。

你好请问你是怎么解决的

[shenqianxiang]

同样的问题。

你解决了吗

[LakeofRain]

我是因为torchvision版本不对，算lpips时vgg网络没下载下来。发自我的荣耀手机-------- 原始邮件 --------发件人： shenqianxiang ***@***.***>日期： 2024年6月26日周三 23:07收件人： graphdeco-inria/gaussian-splatting ***@***.***>抄送： LakeofRain ***@***.***>, Comment ***@***.***>主 题： Re: [graphdeco-inria/gaussian-splatting] Unable to compute metrics for model output/lego (Issue #481) 同样的问题。 你解决了吗 —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***>

[biter0088]

I re-found a new environment on my computer with the torch package, ran pip install torchmetrics and re-executed python metrics.py -m “/home/xiu/Learning/gaussian-splatting/output/train” and was able to output the following results:

# Metric evaluation progress: 100%|█████████████████████████████████████████████████████████████████████████| 38/38 [00:39<00:00, 1.05 s/it]
# here
# SSIM : 0.8076466
# PSNR : 21.7869835
# LPIPS: 0.2143032

[kellofcn]

Same problem here metrics.py does not work. Tried to install torchmetrics but:

• No improvements with pip install
• And the conda install of torchmetrics is messing with my conda environment

This is what I have after a full metrics evaluation : All of the metrics are NaN ...

Sometimes it also crashes :

Traceback (most recent call last):
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/metrics.py", line 150, in <module>
    evaluate(args.model_paths)
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/metrics.py", line 116, in evaluate
    lpipss.append(lpips(renders[idx], gts[idx], net_type='vgg'))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/lpipsPyTorch/__init__.py", line 20, in lpips
    criterion = LPIPS(net_type, version).to(device)
                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/lpipsPyTorch/modules/lpips.py", line 28, in __init__
    self.lin.load_state_dict(get_state_dict(net_type, version))
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/lpipsPyTorch/modules/utils.py", line 17, in get_state_dict
    old_state_dict = torch.hub.load_state_dict_from_url(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/anaconda3/envs/GS/lib/python3.11/site-packages/torch/hub.py", line 770, in load_state_dict_from_url
    return torch.load(cached_file, map_location=map_location, weights_only=weights_only)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/anaconda3/envs/GS/lib/python3.11/site-packages/torch/serialization.py", line 1028, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/anaconda3/envs/GS/lib/python3.11/site-packages/torch/serialization.py", line 1264, in _legacy_load
    typed_storage._untyped_storage._set_from_file(
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

This is what I have executed python metrics.py -m /home/lwk/gaussian-splatting/output/980df71c-a , metric evaluation progress is 0it , do u know why?
Scene: /home/lwk/gaussian-splatting/output/980df71c-a Method: ours_18000 Metric evaluation progress: 0it [00:00, ?it/s] SSIM : nan PSNR : nan LPIPS: nan

[HanYan012]

Same problem here metrics.py does not work. Tried to install torchmetrics but:

• No improvements with pip install
• And the conda install of torchmetrics is messing with my conda environment

This is what I have after a full metrics evaluation : All of the metrics are NaN ...
Sometimes it also crashes :

Traceback (most recent call last):
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/metrics.py", line 150, in <module>
    evaluate(args.model_paths)
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/metrics.py", line 116, in evaluate
    lpipss.append(lpips(renders[idx], gts[idx], net_type='vgg'))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/lpipsPyTorch/__init__.py", line 20, in lpips
    criterion = LPIPS(net_type, version).to(device)
                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/lpipsPyTorch/modules/lpips.py", line 28, in __init__
    self.lin.load_state_dict(get_state_dict(net_type, version))
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/lpipsPyTorch/modules/utils.py", line 17, in get_state_dict
    old_state_dict = torch.hub.load_state_dict_from_url(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/anaconda3/envs/GS/lib/python3.11/site-packages/torch/hub.py", line 770, in load_state_dict_from_url
    return torch.load(cached_file, map_location=map_location, weights_only=weights_only)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/anaconda3/envs/GS/lib/python3.11/site-packages/torch/serialization.py", line 1028, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/anaconda3/envs/GS/lib/python3.11/site-packages/torch/serialization.py", line 1264, in _legacy_load
    typed_storage._untyped_storage._set_from_file(
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

This is what I have executed python metrics.py -m /home/lwk/gaussian-splatting/output/980df71c-a , metric evaluation progress is 0it , do u know why? Scene: /home/lwk/gaussian-splatting/output/980df71c-a Method: ours_18000 Metric evaluation progress: 0it [00:00, ?it/s] SSIM : nan PSNR : nan LPIPS: nan

Same. Did you solve this?

[tapowanliwuyun]

Same. Did you solve this?

[tapowanliwuyun]

solved！please chekck /home/dkcs/.cache/torch/hub/checkpoints/vgg.pth ....

[micki-37]

我是因为torchvision版本不对，算lpips时vgg网络没下载下来。发自我的荣耀手机-------- 原始邮件 --------发件人： shenqianxiang @.>日期： 2024年6月26日周三 23:07收件人： graphdeco-inria/gaussian-splatting @.>抄送： LakeofRain @.>, Comment @.>主 题： Re: [graphdeco-inria/gaussian-splatting] Unable to compute metrics for model output/lego (Issue #481) 同样的问题。 你解决了吗 —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

我也是vgg网络没下完 重新跑metrics就显示算不了了 但并不会重新下vgg的权重 是得重新训练完重新下vgg吗

[LakeofRain]

我换了个torchvision的版本就解决了，没重新训练 在 2025-01-10 10:01:12，"micki-37" ***@***.***> 写道： 我是因为torchvision版本不对，算lpips时vgg网络没下载下来。发自我的荣耀手机-------- 原始邮件 --------发件人： shenqianxiang @.>日期： 2024年6月26日周三 23:07收件人： graphdeco-inria/gaussian-splatting @.>抄送： LakeofRain @.>, Comment @.>主 题： Re: [graphdeco-inria/gaussian-splatting] Unable to compute metrics for model output/lego (Issue #481) 同样的问题。 你解决了吗 —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***> 我也是vgg网络没下完 重新跑metrics就显示算不了了 但并不会重新下vgg的权重 是得重新训练完重新下vgg吗 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>