Evasion Attack Capability for Multi-Label Classification

[MadhuSaran26]

I'm trying to perform a white-box evasion attack (FastGradientMethod) on my pre-trained multi-label classification model. This model has been wrapped in the PyTorchClassifier class and passed as an argument to the FastGradientMethod class. However, I encountered the following error.

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-95-055ea3aa50f4> in <module>
----> 1 images_adv = attack.generate(images.numpy())

6 frames
/usr/local/lib/python3.9/dist-packages/torch/nn/functional.py in binary_cross_entropy_with_logits(input, target, weight, size_average, reduce, reduction, pos_weight)
   3158 
   3159     if not (target.size() == input.size()):
-> 3160         raise ValueError("Target size ({}) must be the same as input size ({})".format(target.size(), input.size()))
   3161 
   3162     return torch.binary_cross_entropy_with_logits(input, target, weight, pos_weight, reduction_enum)

ValueError: Target size (torch.Size([16])) must be the same as input size (torch.Size([16, 9]))

In this, 16 is the batch size and 9 is the number of output classes in the model.

The following code produced this error.

classifier = PyTorchClassifier(
    model=vgg,
    clip_values=(0, 255),
    loss=nn.BCEWithLogitsLoss(),
    optimizer=optimizer,
    input_shape=(3, 256, 256),
    nb_classes=9,
    device_type='cpu'
)

attack = evasion.FastGradientMethod(estimator=classifier, eps=0.2)

images_adv = attack.generate(images.numpy())

I'd like to know whether this toolbox can be used for a multi-label classification problem. If yes, then please kindly direct me to an example execution, or at least let me know what are the changes that I should make to my code. Otherwise, please provide the solution to this problem.

In addition, I wanted to point out that it's not feasible to create combined labels as there are multiple combinations, which could be possible in a scenario.

[beat-buesser]

Hi @MadhuSaran26 Could you please share a code example that reproduces your observation?

[MadhuSaran26]

Hi @beat-buesser, I have added the code that produces the error in the issue description itself. I have fine-tuned a VGG19 network that's pre-trained on ImageNet weights. This network performs a multi-label classification, which outputs 9 one-hot encoded labels. I'm loading the images and the respective labels in batches of 16.

Then, I'm performing the following action, which yields the described error.

`classifier = PyTorchClassifier(
model=vgg,
clip_values=(0, 255),
loss=nn.BCEWithLogitsLoss(),
optimizer=optim.Adam(vgg, lr=0.0001),
input_shape=(3, 256, 256),
nb_classes=9,
device_type='cpu'
)

attack = evasion.FastGradientMethod(estimator=classifier, eps=0.2)

images_adv = attack.generate(images.numpy())`

[beat-buesser]

Hi @MadhuSaran26 Thank you, can you also include the code that imports tools, data and model? I'd be interested to try it out. Maybe, can you share it in a Google Colab notebook?

[MadhuSaran26]

Hi @beat-buesser, unfortunately I'm not able to share the code file. However, I was trying something similar to what's present in the following link - https://debuggercafe.com/multi-label-image-classification-with-pytorch-and-deep-learning/

As you can see in the given example, an image can have multiple labels associated with it, which are given to the model as one-hot encoded values. In these cases, the PyTorchClassifier method if not supported I believe. Can you please help me with this issue?

Also, I'm sorry for the delayed response.

Thanks,
Madhumitha

[MadhuSaran26]

Hi @beat-buesser, any luck on this?

Perhaps, I could compile the code into a single file and attach it here for your convenience. Please let me know.