Inconsistent Number of Predictions

[agp-ka32]

Hi @ppwwyyxx ,

====Some background:====
I am able to run a pre-trained model on any input image to get mask predictions as well as the bounding boxes. I have set my model in eval mode, as required.

In doing so, I also display the output of my model as below:

output = model([inputs])
print('output instances are:', output[0]['instances'])

This gives me 7 class predictions for different objects in my image, i.e, num_instances = 7.
So far so good.

====My question:====
I am also getting the mask features from the same pre-trained model used above, using the following code:

features = model.backbone(images.tensor)
proposals, _ = model.proposal_generator(images, features)
instances, _ = model.roi_heads(images, features, proposals)

(I have omitted the required two lines to get mask features for brevity.)

The num_instances here is 5, which is different from when I run the model without any incisions, as described earlier (it was 7 there).

I am curious to know why this inconsistency.

Can you clarify? Thanks.

[agp-ka32]

@ppwwyyxx Any pointers?

[AhmedMasryKU]

I found a solution.By inspecting the DefaultPredictor source code, I found that we need to do an additional preprocessing step for the image. You can use the following code snippet to prerpo

import detectron2.data.transforms as T
aug = T.ResizeShortestEdge(
    [cfg.INPUT.MIN_SIZE_TEST, cfg.INPUT.MIN_SIZE_TEST], cfg.INPUT.MAX_SIZE_TEST
)
image = aug.get_transform(image).apply_image(image)
image = torch.as_tensor(image.astype("float32").transpose(2, 0, 1))
inputs = [{"image": image, "height": height, "width": width}]
#run model
with torch.no_grad():
    images = model.preprocess_image(inputs)  # don't forget to preprocess
    features = model.backbone(images.tensor)
...
...

[agp-ka32]

Thanks, Ahmed. This fixes the issue.

[agp-ka32]

Also, I see that the detected bounding box coordinates are on the transformed (scaled) version of the input image.

How to get the box coordinates on the original image?