Feature Collision Example(Notebook) does not perform as expected

[Rock910]

Describe the bug
Following the exact steps in the feature collision notebook, I am unable to reproduce the result in the notebook.
After the line:
fig = plt.imshow(target_instance[0])

print('true_class: ' + target_class)
print('predicted_class: ' + class_descr[np.argmax(classifier.predict(target_instance), axis=1)[0]])

true_class: bird
predicted_class: frog is what we expect, but in my case they are both bird.
I am unable to set_learning_phase(True) as KerasClassifier does not have this attribute.
Looking deeper into KerasClassifier this seems to be the case, and I am not sure how to work around.
To Reproduce
Steps to reproduce the behavior:

1. Go to adversarial-robustness-toolbox/notebooks/poisoning_attack_feature_collision.ipynb
   2). Perform all the steps
   3). See error, happens after running set_learning_phase(True)
   classifier.set_learning_phase(True)
   print(x_train.shape)
   print(base_instances.shape)
   adv_train = np.vstack([x_train, poison])
   adv_labels = np.vstack([y_train, poison_labels])
   classifier.fit(adv_train, adv_labels, nb_epochs=5, batch_size=4)

Expected behavior
A clear and concise description of what you expected to happen.
true_class: bird
predicted_class: frog is what we expect, but in my case they are both bird.

System information (please complete the following information):

• OS: Linux machine
• Python version: 3.8.15
• ART version or commit number: 1.12.2
• TensorFlow / Keras / PyTorch / MXNet version: Keras

[beat-buesser]

Hi @Rock910 Have you tried running the notebook multiple times? Does observation persist with multiple runs?

[Rock910]

Hi,
I've tried to run it 2-3 times with no luck.
I appreciate your help with this.

(Edited): I ran it a couple more times and have success.
Thank you very much, why is this the case?

[beat-buesser]

The notebook trains a neural network, it's random initialization could have an influence. I would experiment with the attack parameters (e.g. ma_iter, etc.) to make the attack stronger and more consistent and to avoid random effects. The notebooks are sometimes trade-offs between showing how an attack works but with parameters that don't require too long run times.