two stage 3D multiclass segmentation pipeline

[YingdiZhang]

Thanks for the great job. I met a problem when using monai.
I want to achieve 3d image multiclass segmentation in a two stage pipeline： 1 foreground segmentation 2 multiclass segmentation
I have trained a foreground segmentation network and a multiclass segmentation network（crop foreground firstly by using MONAI.
But I dont know how to combine these two network together in an inference loop. The toturials are all based on one network segmentation.

[KumoLiu]

Hi @YingdiZhang, if it's a two-stage pipeline, then you can just put the input into the first one and the output into the second one.

out1 = model1(input)
out = model2(out1)

Thanks.

[YingdiZhang]

Thanks for your reply.
In my case, the out1 cant be the input of model2, it is used to crop the foreground of image. I write a code script，but always make an error.
`
The code is:

 for (idx, test_data) in enumerate(tqdm(self.test_loader)):
       inputs = test_data["image"].to(device)
       label = test_data["label"].to(device)
        test_data["pred_foreground_binary"] = sliding_window_inference(
                         inputs, roi_size_stage1, sw_batch_size, self.model_stage1, 
                         overlap=overlap_stage1,
                         mode='gaussian',
                         device=(torch.device('cpu') if test_id in cpu_data else device)
                       ) 
          test_data["pred_foreground_binary"] = test_data["pred_foreground_binary"][0] \
                if isinstance(test_data["pred_foreground_binary"], tuple) else test_data["pred_foreground_binary"]
           self.post_trans_stage1 = Compose(
                [
                    AsDiscreted(keys="pred_foreground_binary", argmax=True),  
                    CropForegroundd(keys=["image"],
                                    source_key="pred_foreground_binary",
                                    margin=10
                                    ),
                    EnsureChannelFirstd(keys=["image", "label"]),
                ]
            )
            test_data = [self.post_trans_stage1(i) for i in decollate_batch(test_data)]
            test_data = test_data[0]
            test_data["pred"] = sliding_window_inference(
                test_data["image"].to(device),
                roi_size_stage2,
                sw_batch_size,
                self.model_stage2,
                overlap=overlap_stage2,
                mode='gaussian',
                device=(torch.device('cpu') if test_id in cpu_data else device)
                ) 
              test_data["pred"] = test_data["pred"][0] \
                if isinstance(test_data["pred"], tuple) else test_data["pred"]

`
I dont know how to restore test_data["pred"] back to the size of test_data["label"] and save the segmentation result

[KumoLiu]

You can try inverse on the pred.
https://github.com/Project-MONAI/tutorials/blob/main/modules/inverse_transforms_and_test_time_augmentations.ipynb

[YingdiZhang]

Thanks for your reply. I added a Inverse transform：
`
self.post_trans_pred = Compose(
[
EnsureTyped(keys="pred"),
Invertd(
keys="pred",
transform=self.post_trans_stage1,
orig_keys="image",
meta_keys="pred_meta_dict",
orig_meta_keys="image_meta_dict",
meta_key_postfix="meta_dict",
nearest_interp=False,
to_tensor=True # convert to PyTorch Tensor after inverting
),
AsDiscreted(keys="pred", argmax=True),
SaveImaged(keys="pred", meta_keys="pred_meta_dict", output_dir=self.pred_output_folder,
output_postfix="seg", resample=False, output_ext='.nii.gz',
separate_folder=False)
]
)
test_outputs = [self.post_trans_pred(i) for i in decollate_batch(test_data)]

`
And got an error：
Traceback (most recent call last):
File "/home/zyd/anaconda3/envs/monai1.1/lib/python3.9/site-packages/monai/transforms/transform.py", line 102, in apply_transform
return _apply_transform(transform, data, unpack_items)
File "/home/zyd/anaconda3/envs/monai1.1/lib/python3.9/site-packages/monai/transforms/transform.py", line 66, in _apply_transform
return transform(parameters)
File "/home/zyd/anaconda3/envs/monai1.1/lib/python3.9/site-packages/monai/transforms/croppad/dictionary.py", line 318, in inverse
d[key] = self.cropper.inverse(d[key])
File "/home/zyd/anaconda3/envs/monai1.1/lib/python3.9/site-packages/monai/transforms/croppad/array.py", line 885, in inverse
transform = self.get_most_recent_transform(img)
File "/home/zyd/anaconda3/envs/monai1.1/lib/python3.9/site-packages/monai/transforms/inverse.py", line 188, in get_most_recent_transform
self.check_transforms_match(all_transforms[-1])
KeyError: -1

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/zyd/anaconda3/envs/monai1.1/lib/python3.9/site-packages/monai/transforms/transform.py", line 102, in apply_transform
return _apply_transform(transform, data, unpack_items)
File "/home/zyd/anaconda3/envs/monai1.1/lib/python3.9/site-packages/monai/transforms/transform.py", line 66, in _apply_transform
return transform(parameters)
File "/home/zyd/anaconda3/envs/monai1.1/lib/python3.9/site-packages/monai/transforms/post/dictionary.py", line 701, in call
inverted = self.transform.inverse(input_dict)
File "/home/zyd/anaconda3/envs/monai1.1/lib/python3.9/site-packages/monai/transforms/compose.py", line 184, in inverse
data = apply_transform(t.inverse, data, self.map_items, self.unpack_items, self.log_stats)
File "/home/zyd/anaconda3/envs/monai1.1/lib/python3.9/site-packages/monai/transforms/transform.py", line 129, in apply_transform
raise RuntimeError(f"applying transform {transform}") from e
RuntimeError: applying transform <bound method Cropd.inverse of <monai.transforms.croppad.dictionary.CropForegroundd object at 0x7fb94c832fd0>>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/data/zyd/beifen_xinsuoqu_/MedSeg/MedSeg/test/Task966Tester.py", line 1239, in
t966_tester.test(
File "/data/zyd/beifen_xinsuoqu_/MedSeg/MedSeg/test/Task966Tester.py", line 286, in test
test_outputs = [self.post_trans_pred(i) for i in decollate_batch(test_data)]
File "/data/zyd/beifen_xinsuoqu_/MedSeg/MedSeg/test/Task966Tester.py", line 286, in
test_outputs = [self.post_trans_pred(i) for i in decollate_batch(test_data)]
File "/home/zyd/anaconda3/envs/monai1.1/lib/python3.9/site-packages/monai/transforms/compose.py", line 174, in call
input_ = apply_transform(transform, input, self.map_items, self.unpack_items, self.log_stats)
File "/home/zyd/anaconda3/envs/monai1.1/lib/python3.9/site-packages/monai/transforms/transform.py", line 129, in apply_transform
raise RuntimeError(f"applying transform {transform}") from e
RuntimeError: applying transform <monai.transforms.post.dictionary.Invertd object at 0x7fb94c7bda30>
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 2264085) of binary: /home/zyd/anaconda3/envs/monai1.1/bin/python
Traceback (most recent call last):
File "/home/zyd/anaconda3/envs/monai1.1/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/zyd/anaconda3/envs/monai1.1/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/zyd/anaconda3/envs/monai1.1/lib/python3.9/site-packages/torch/distributed/launch.py", line 195, in
main()
File "/home/zyd/anaconda3/envs/monai1.1/lib/python3.9/site-packages/torch/distributed/launch.py", line 191, in main
launch(args)
File "/home/zyd/anaconda3/envs/monai1.1/lib/python3.9/site-packages/torch/distributed/launch.py", line 176, in launch
run(args)
File "/home/zyd/anaconda3/envs/monai1.1/lib/python3.9/site-packages/torch/distributed/run.py", line 753, in run
elastic_launch(
File "/home/zyd/anaconda3/envs/monai1.1/lib/python3.9/site-packages/torch/distributed/launcher/api.py", line 132, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/home/zyd/anaconda3/envs/monai1.1/lib/python3.9/site-package