Paralellize Sliding Window Inference on CPU

[pcanas]

I am using

from monai.inferers import sliding_window_inference

For our use case, we need to run this inference on CPU, and performance is significantly worse (x10 times worse) than using GPU. Since the algorithm could be parallelized, I was wondering how we can achieve parallelization for this function.

[KumoLiu]

Hi, @pcanas

Currently, MONAI does not provide parallelization for the computation on CPU, you can try joblib package to accelerate the computation, more details can be found in https://joblib.readthedocs.io/en/latest/generated/joblib.Parallel.html .

Thanks

[pcanas]

Hi @KumoLiu thanks for replying.
I do not see how could I use joblib in this scenario, if all the parallelizable logic is inside the MONAI package. Could you further develop your answer?

Thank you!

[KumoLiu]

Hi @pcanas, to parallelize the MONAI's sliding_window_inference function using joblib, you can follow these steps:

1. Import the required modules, and define the sliding_window_inference function with the desired parameters:

   import joblib
   from monai.inferers.utils import sliding_window_inference
   
   def run_inference(data):
       # Your code to load and preprocess the data goes here
       result = sliding_window_inference(data, roi_size=(64, 64, 64), sw_batch_size=2)
       # Your code to process the result goes here
       return result
   

2. Prepare a list of input data that you want to process in parallel:

   data_list = [...]  # List of input data
   

3. Use joblib to parallelize the execution of run_inference function on the data list:

   results = joblib.Parallel(n_jobs=-1)(joblib.delayed(run_inference)(data) for data in data_list)
   

   Here, n_jobs=-1 specifies that you want to use all available CPU cores for parallel processing. Adjust this value according to your system's capabilities.

By utilizing joblib's Parallel function, the run_inference function will be executed in parallel on multiple input data, leading to faster processing times.
Thanks!

[pcanas]

Hi @KumoLiu and thanks for taking your time to asking so comprehensively.

While I appreciate your answer, it does not really solve my problem, since I wanted to parallelize jobs for one inference and not use parallelization to run multiple inferences at the same time. I hope that there is a solution for this using joblib.

Regards

[KumoLiu]

Hi @pcanas, if you want to do parallelize inside sliding_window_inference, you may need to modify the code manually. Or you can increase your sw_batch_size.
BTW, if you have a limited GPU resource, you can also try SlidingWindowInfererAdapt which automatically switch to buffered and then to CPU stitching, when OOM on GPU.

MONAI/monai/inferers/inferer.py

Line 519 in 17c1e3a

class SlidingWindowInfererAdapt(SlidingWindowInferer):

Hope it helps, thanks!

[pcanas]

Absolutely @KumoLiu Thanks for the support.