Run demo.py on multiple GPUs

[ashnair1]

Given a directory of images, I wanted to run model inference using multiple GPUs. Using the run_on_video function as a template, I wrote the run_on_images function for VisualisationDemo as follows:

def run_on_images(self, images,):
    
    vis_kwargs = {"metadata": self.metadata, "instance_mode": self.instance_mode}
    
    def process_predictions(img, predictions, kwargs):
        visualizer = Visualizer(img, **kwargs)
        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) #img[:, :, ::-1]
        if "instances" in predictions:
            instances = predictions["instances"].to(self.cpu_device)
            vis_output = visualizer.draw_instance_predictions(predictions=instances)
        # Converts Matplotlib RGB format to OpenCV BGR format
        vis_output = cv2.cvtColor(vis_output.get_image(), cv2.COLOR_RGB2BGR)
        return predictions, vis_output

    imgs = (i for i in images)

    if self.parallel:
        
        buffer_size = self.predictor.default_buffer_size
        im_data = deque()

        for cnt, im in enumerate(imgs):
            im_data.append(im)
            self.predictor.put(im)

            if cnt >= buffer_size:
                im = im_data.popleft()
                preds = self.predictor.get()
                yield process_predictions(im, preds, vis_kwargs)
                
        while len(im_data):
            im = im_data.popleft()
            preds = self.predictor.get()
            yield process_predictions(im, preds, vis_kwargs)

    else:
        for im in imgs:
            yield process_predictions(im, self.predictor(im), vis_kwargs)

It seems to be working as expected but at the end of the iteration just before the function terminates, I get this warning:

[W CudaIPCTypes.cpp:21] Producer process has been terminated before all shared CUDA tensors released. See Note [Sharing CUDA tensors]

Does anyone have any idea how to fix this?

[fenghuoxiguozu]

when use origin run_on_image method,the multi gpus inference time more than single gpu time. do you some question?