Skip to content

[Feature Request] 3D support for PanopticQualityMetric + provide option to return confusion matrix #5702

@neuronflow

Description

@neuronflow

** Edit ** Feature request:

  • 3D support for PanopticQualityMetric
  • provide option to return confusion matrix

Describe the bug
Not sure whether one should report bugs for weekly versions?

I am playing around with PanopticQualityMetric.

The metric_name is ignored, I always get the same output, no matter if I use:

    metric_name=("pq")
    metric_name=("sq")
    metric_name=("rq")
    metric_name=("pq", "sq", "rq")

Also there is no option to turn on the confusion matrix output already, correct?

When I turn it on in the code I get a tensor like:

score1: tensor([153.0000,  24.0000,  21.0000, 139.7191])

If I am not mistaken these values represent the following?

tp, fp, fn, iou_sum = f[..., 0], f[..., 1], f[..., 2], f[..., 3]

feature request: I believe there should also be an option to directly compute mean iou as it is done in the manuscript.

To Reproduce
Compute PanopticQualityMetric with different metric_name parameters.
Environment

python -c 'import monai; monai.config.print_debug_info()'

python -c 'import monai; monai.config.print_debug_info()'

================================
Printing MONAI config...

MONAI version: 1.1.dev2250
Numpy version: 1.23.5
Pytorch version: 1.13.0+cu117
MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False
MONAI rev id: 1d25ea145532319feb57647797ed3a8c2e7e9eb4
MONAI file: /home/florian/miniconda3/envs/emcaps/lib/python3.10/site-packages/monai/init.py

Optional dependencies:
Pytorch Ignite version: NOT INSTALLED or UNKNOWN VERSION.
Nibabel version: 4.0.2
scikit-image version: 0.19.3
Pillow version: 9.3.0
Tensorboard version: 2.11.0
gdown version: 4.5.4
TorchVision version: NOT INSTALLED or UNKNOWN VERSION.
tqdm version: 4.64.1
lmdb version: NOT INSTALLED or UNKNOWN VERSION.
psutil version: 5.9.4
pandas version: 1.5.1
einops version: NOT INSTALLED or UNKNOWN VERSION.
transformers version: NOT INSTALLED or UNKNOWN VERSION.
mlflow version: NOT INSTALLED or UNKNOWN VERSION.
pynrrd version: NOT INSTALLED or UNKNOWN VERSION.

For details about installing the optional dependencies, please visit:
https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies

================================
Printing system config...

System: Linux
Linux version: Ubuntu 20.04.5 LTS
Platform: Linux-5.15.0-50-generic-x86_64-with-glibc2.31
Processor: x86_64
Machine: x86_64
Python version: 3.10.8
Process name: python
Command: ['python', '-c', 'import monai; monai.config.print_debug_info()']
Open files: [popenfile(path='/home/florian/.vscode-server/data/logs/20221211T175743/remoteagent.log', fd=19, position=20132, mode='a', flags=33793), popenfile(path='/home/florian/.vscode-server/data/logs/20221211T175743/ptyhost.log', fd=20, position=4094, mode='a', flags=33793), popenfile(path='/home/florian/.vscode-server/bin/5235c6bb189b60b01b1f49062f4ffa42384f8c91/vscode-remote-lock.florian.5235c6bb189b60b01b1f49062f4ffa42384f8c91', fd=99, position=0, mode='w', flags=32769)]
Num physical CPUs: 16
Num logical CPUs: 32
Num usable CPUs: 32
CPU usage (%): [32.1, 20.7, 16.2, 17.0, 45.5, 30.8, 36.7, 36.4, 40.6, 17.9, 40.6, 23.4, 18.9, 15.2, 16.0, 86.0, 16.8, 30.8, 18.9, 18.1, 22.6, 34.3, 32.1, 34.0, 56.5, 35.5, 15.2, 27.4, 18.5, 16.8, 16.0, 30.8]
CPU freq. (MHz): 2008
Load avg. in last 1, 5, 15 mins (%): [11.3, 14.0, 14.8]
Disk usage (%): 35.1
Avg. sensor temp. (Celsius): UNKNOWN for given OS
Total physical memory (GB): 125.7
Available memory (GB): 114.8
Used memory (GB): 9.5

================================
Printing GPU config...

Num GPUs: 2
Has CUDA: True
CUDA version: 11.7
cuDNN enabled: True
cuDNN version: 8500
Current device: 0
Library compiled for CUDA architectures: ['sm_37', 'sm_50', 'sm_60', 'sm_70', 'sm_75', 'sm_80', 'sm_86']
GPU 0 Name: NVIDIA RTX A5000
GPU 0 Is integrated: False
GPU 0 Is multi GPU board: False
GPU 0 Multi processor count: 64
GPU 0 Total memory (GB): 23.7
GPU 0 CUDA capability (maj.min): 8.6
GPU 1 Name: Quadro RTX 8000
GPU 1 Is integrated: False
GPU 1 Is multi GPU board: False
GPU 1 Multi processor count: 72
GPU 1 Total memory (GB): 47.5
GPU 1 CUDA capability (maj.min): 7.5

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions