ADA-Steal: Medical Multimodal Model Stealing Attacks via Adversarial Domain Alignment

🔥 Official implementation of "Medical Multimodal Model Stealing Attacks via Adversarial Domain Alignment" (AAAI-2025 oral)

Installation

Environment

To set up the environment, follow these steps:

1. Create a new Conda environment

conda env create -f environment.yml

2. Activate the new environment

conda activate ada_steal

3. Configure API Keys

Create an .env file containing huggingface token and path to save models.

HUGGINGFACE_TOKEN=hf_xxxxxxx
HF_CACHE_DIR=$PATH/TO/CACHE/DIR
OPENAI_API_KEY=xxxxx

Models

• Victim Model: CheXagent-8b
• Attack Model: IDEFICS-9b

Prepare the Data

We use chest X-ray datasets (MIMC-CXR and IU X-Ray) as well as one natural image dataset (i.e., CIFAR-100) in our paper.

• For MIMIC-CXR, you can download the MIMIC-CXR-JPG dataset from here (PhysioNet permission required) and then put the files in data/mimic_cxr.
• For IU X-Ray, you can down load the dataset from here and then put the files in data/iu_xray.

data/
│
├── iu_xray/
│   ├── images/               # IU X-Ray image files
│   └── annotation.json       # Original IU X-Ray annotations
│
├── mimic-cxr/
│   ├── test/                 # MIMIC-CXR test split images
│   ├── train/                # MIMIC-CXR training split images
│   └── mimic-test.json       # Annotation file for the test set
│
├── oracle_texts/
│   ├── abnormalities_zephyr.json 
│   ├── no_findings_zephyr.json
│   ├── abnormalities_gpt4.json
│   └── no_findings_gpt4.json   
│
└── ini_data.pkl

Run Experiments

Then start with:

# Query the victim model with cifar training images to get initial attack data with (cifar-image, victim report) pairs
python create_data.py --budget 500

# Launch ADA-Steal attack to steal victim model on iu_xray
python main_attack.py \
    --train_path "data/ini_data.pkl" \
    --image_dir "data/iu_xray/images" \
    --test_path "data/iu_xray/annotation.json" \
    --attack_model IDEFICS \
    --model_checkpoint HuggingFaceM4/idefics-9b \
    --max_seq_length 200 \
    --budget 50 \
    --epochs 5 \
    --batch_size 8 \
    --lr 1e-5 \
    --lr_scheduler constant \
    --criteria last \
    --num_rounds 3 \
    --oracle_switch off \
    --resume on \
    --seed 7580 \
    --save_dir "adversarial/outputs/" \
    --save_record "results/"

You could refer to all_exp.sh for different experiment's commands.

Citations

If you use or extend our work, please cite our paper at AAAI-2025

@inproceedings{shen2025medical,
  title={Medical multimodal model stealing attacks via adversarial domain alignment},
  author={Shen, Yaling and Zhuang, Zhixiong and Yuan, Kun and Nicolae, Maria-Irina and Navab, Nassir and Padoy, Nicolas and Fritz, Mario},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)},
  year={2025}
}

License

This project is open-sourced under the AGPL-3.0 license. See the LICENSE file for details.

For a list of other open source components included in this project, see the file 3rd-party-licenses.txt.

Purpose of the project

This software is a research prototype, solely developed for and published as part of the publication cited above.

Contact

Please feel free to open an issue or contact personally if you have questions, need help, or need explanations. Don't hesitate to write an email to the following email address: yaling.shen@tum.de