A confusion about defense dataset

[wizaaaard]

Dear author, thank you for your code and creative ideas, they have left a deep impression on me. I have a question regarding your code: In core.dataset.py, in each dataset class, you split the test_dataset into defense_test_dataset and defense_val_dataset. However, you do not use defense_val_dataset anywhere, and why don't you directly use test_dataset for testing? I am looking forward to your reply. Thank you again.

[hkucs-kachow]

Thank you for your question. We are thrilled that Imperio has left a deep impression on you.

It was supposed to be used by defenses to learn their hyperparameters. If you focus on attacks and want more data samples for testing, you can disable the split and use the entire test dataset.

[wizaaaard]

Could you please clarify whether the testing metrics provided in the paper were tested on the complete test dataset or on the defense_dataset in the current code?