Tensor Type Error when Using Robust DPatch

[dominic-simon]

Hello, I am getting the following error when trying to generate the Robust DPatch:

RuntimeError: Input type (torch.cuda.DoubleTensor) and weight type (torch.cuda.FloatTensor) should be the same

The image data type and model data type are not the same. I pass the image as a numpy array to the generation function, so I have no access to the image as a torch tensor. I tried forcing the model to use the DoubleTensor type, but it doesn't seem to work.

I have the normal DPatch working, but I would prefer to use the Robust DPatch. I'm not sure where I'm going wrong, so any help is appreciated. Let know if I need to provide any additional information. Here is my code for reference:

from art.attacks.evasion import DPatch, RobustDPatch
from art.estimators.object_detection import PyTorchFasterRCNN

import torchvision.models.detection as models
from torchvision.datasets import CocoDetection

import numpy as np

model = models.fasterrcnn_resnet50_fpn_v2(weights='FasterRCNN_ResNet50_FPN_V2_Weights.DEFAULT')
frcnn = PyTorchFasterRCNN(model, channels_first=False, device_type='gpu')

rdpatch = RobustDPatch(frcnn, patch_shape=(32, 32, 3), patch_location=(50, 50),
                       crop_range=(50, 50), brightness_range=[0.7, 1.0],
                       rotation_weights=(1, 1, 1, 1), sample_size=32, learning_rate=1.0,
                       max_iter=1000, batch_size=1, targeted=True, verbose=True)

ms_coco = CocoDetection('../datasets/MSCOCO/val2017', '../datasets/MSCOCO/annotations/instances_val2017.json')

for i in range(32):
    image = np.array(ms_coco[i][0])
    image = np.expand_dims(image, axis=0)

    y = [{'boxes':np.array([[50, 50, 82, 82]]),
          'labels':np.array([[5]]),
          'scores':np.array([[0.75]])}]
    _ = rdpatch.generate(image, y)