Adversarial training mnist

[thungp]

Ref:
(a)
https://github.com/Trusted-AI/adversarial-robustness-toolbox/blob/main/notebooks/adversarial_training_mnist.ipynb

(b) #367

Could someone tell me, if there is Adversarial training example for this notebook that uses TF 2.x libraries?

Additionally, has anyone by chance adopted this sample code to work
with Resnet50 and/or inceptionv3 model?

I made an attempt to upgrade the code to use TF 2.x packages.
Summary of some of he key changes attempted

https://colab.research.google.com/drive/1vxdj8kYrZI9d6SDdfubGA4m5U_EgqdbR?usp=sharing
But getting this error.

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-19-70de71d5aabd> in <module>
      7 trainer = AdversarialTrainer(robust_classifier, attacks, ratio=1.0)
      8 # dir(trainer)
----> 9 trainer.fit(x_train, y_train, nb_epochs=83, batch_size=50)

~/.conda/envs/art/lib/python3.6/site-packages/art/defences/trainer/adversarial_trainer.py in fit(self, x, y, batch_size, nb_epochs, **kwargs)
    243 
    244                 # Fit batch
--> 245                 self._classifier.fit(x_batch, y_batch, nb_epochs=1, batch_size=x_batch.shape[0], verbose=0, **kwargs)
    246                 attack_id = (attack_id + 1) % len(self.attacks)
    247 

~/.conda/envs/art/lib/python3.6/site-packages/art/estimators/classification/classifier.py in replacement_function(self, *args, **kwargs)
     69                 if len(args) > 0:
     70                     args = tuple(lst)
---> 71                 return fdict[func_name](self, *args, **kwargs)
     72 
     73             replacement_function.__doc__ = fdict[func_name].__doc__

~/.conda/envs/art/lib/python3.6/site-packages/art/estimators/classification/tensorflow.py in fit(self, x, y, batch_size, nb_epochs, **kwargs)
    946         if self._train_step is None:
    947             raise TypeError(
--> 948                 "The training function `train_step` is required for fitting a model but it has not been " "defined."
    949             )
    950 

TypeError: The training function `train_step` is required for fitting a model but it has not been defined.

I'm only speculating at this point, before I potentially dive deeper,
could it be that the AdversarialTrainer class needs to get upgraded to TF2.x? (maybe leveraging https://www.tensorflow.org/guide/upgrade)?
or hopefully maybe some error on my part?

-Peter

[beat-buesser]

Hi @thungp To train a model with TensorFlowV2Classifier you need to define a function defining the training step for the model and provide it to argument train_step of TensorFlowV2Classifier similar as in this example script: https://github.com/Trusted-AI/adversarial-robustness-toolbox/blob/main/examples/get_started_tensorflow_v2.py

[beat-buesser]

Hi @thungp Thank you very much for sharing your work! Let me try to answer the questions first:

2. I couldn't find any scripts anymore that trained the model used in this notebook. It looks like the original contributor only shared the notebook and the model.
3. Yes, the figure in the notebook should look more like Figure 6 of https://arxiv.org/pdf/1706.06083.pdf. I think it is masking gradients because it looks to have a very high or even perfect confidence which results in a categorical entropy loss that is numerically zero and therefore also its loss gradients are numerically zero and that's why the attack doesn't work, but other attack approaches would work on this model.
4. Yes, both frameworks should achieve the same results.

• Yes, usually all layers should be trainable for adversarial training if starting training from scratch. ART calls the respective frameworks by setting the most likely defaults for trainability depending on the tasks and depending on if the framework requires it or not using the optional argument training_mode in the supporting estimator's methods. For special task-specific models like ASR ART's estimators contain special internal methods to set batch-norm layers, etc. correctly.

The image above looks good, there is no more gradient masking as all images have perturbations. What are the benign and adversarial accuracies on the training and test sets?

Would you be interested to share your results to update the notebook and fix #1298?

[beat-buesser]

@thungp

About the issue of slow iterations, what is your batch-size during training and attack?

About the gradient masking in the first image of MNIST images, you could calculate the cross entropy loss for images that show perturbations and for images that don't, you should see that the latter is numerically very low.

[thungp]

@beat-buesser ,

Will get back to you on your other question and responses, but, just wanted to post the result of the TF 2.3
results across the range of EPS values:

The originally classifier results seem a little too good and is tracking the robust classifier curve a little too closely IMHO.

I feel, like I may need to go over my script a little closely.

[thungp]

@thungp

About the issue of slow iterations, what is your batch-size during training and attack?

About the gradient masking in the first image of MNIST images, you could calculate the cross entropy loss for images that show perturbations and for images that don't, you should see that the latter is numerically very low.

Batch_size = 50
trainer.fit(x_train, y_train, nb_epochs=83, batch_size=50) # baseline

[beat-buesser]

@thungp I agree with your opinion on the last post figure. Would you be able to share an update version of your notebook to try it out?

One thing we noticed previously is that adversarial training depends strongly on the hyper-parameters. I would recommend to start with exactly the same parameter values as in https://arxiv.org/pdf/1706.06083.pdf.

[thungp]

@beat-buesser ,
https://colab.research.google.com/drive/1TqhB_Gsc3Q23rh1wHmlRL296__l9BeKn?usp=sharing

1. I looked at the article, and tried to match the pareameter values. Not sure if you have time to double check.
2. In the original example art, there was a comment about the following:
   "# We had performed this before, starting with a randomly intialized model.

Adversarial training takes about 80 minutes on an NVIDIA V100.

The resulting model is the one loaded from mnist_cnn_robust.h5 above."

I guess I must have miissed that, because as you can see in my example, I assumed that I would be starting out
with a pretrained model and when changing the configuration of one of the dense layers, I would want to try and
treat it like transer learning. and would want to have some weights to the model before attemppting to train it further
with with adversarial samples.

If the whole model is supposed to be randomly1 initialized, it then begs the question, how is it randomly initialized?
Also, ifyou have any exmaples of how to save the model that is wrapped after adversarially trained,
please feel free to send some sample code.
robust_classifier = modified_classifier

Also training is takng almost 22 hours on Google Colab, vice the 80 minutes that the example is saying which is also making
me question stuff.

[beat-buesser]

Hi @thungp Thank you for sharing your results. I have tried to open the Colab notebook, but I had to request access first, could you please grant access?

Is eps in infinity norm?

I think the red line looks quite similar as the blue line in Figure 6a of Madry et al. (https://arxiv.org/pdf/1706.06083.pdf).

The blue line in your plot should converge to zero, and decrease much earlier. Can you check if gradient masking occurs?

About the training time, what is your batch size? How busy is the GPU during the 22 hours?

Most users access to he estimator property model (e.g. robust_classifier.model) and use the tools of the corresponding ML framework to save the model.

[thungp]

@beat-buesser
I think I fixed the link to allow anyone with the link to be able to view the notebook.
https://colab.research.google.com/drive/1TqhB_Gsc3Q23rh1wHmlRL296__l9BeKn?usp=sharing

Else, I think I would need to get your email address to to grant you specific access.
When you get the chance, please try the link [https://colab.research.google.com/drive/1TqhB_Gsc3Q23rh1wHmlRL296__l9BeKn?usp=sharing] again.

[beat-buesser]

@thungp Thank you! As you have mentioned above, to get started with adversarial training it would definitely be easier if you don't have the frozen layers. I would recommend to exactly reproduce the experiments of Madry et al. (https://arxiv.org/pdf/1706.06083.pdf, their experiment code is available on GitHub too). For these experiments, I also would not work with mnist_cnn_original.h5 at all and start to implement the model completely from scratch instead, it will not require much code and be easier to initialise, etc.