FedAdapter (old name: AdaFL) is an adapter-based efficient FedNLP framework for accelerating federated learning (FL) in natural language processing (NLP).
FedAdapter is built atop FedNLP beta (commit id: 27f3f97), the document file and instruction could be found in README_FedNLP.md
Install docker in your machine. Then run the following command to build the docker image.
docker pull caidongqi/adafl:1.1.0
docker run -it --gpus all --network host caidongqi/adafl:1.1.0 bashWe recommand using VSCode docker extension to develop in the docker container.
After git clone-ing this repository, please run the following command to install our dependencies.
conda create -n fednlp python=3.7
conda activate fednlp
pip3 install torch==1.10.0+cu113 torchvision==0.11.1+cu113 torchaudio==0.10.0+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html
pip install -r requirements.txt
# some nail wheels, we install them manually
conda install mpi4py=3.0.3=py37hf046da1_1
conda install six==1.15.0
cd FedML; git submodule init; git submodule update; cd ../;
Our system is implemented on:
Linux Phoenix22 5.4.0-122-generic #138~18.04.1-Ubuntu SMP Fri Jun 24 14:14:03 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Each simulated client needs ~2GB GPU memory, ~10GB RAM memory.
cd data
# remember to modify the path in download_data.sh and download_partition.sh
bash download_data.sh
bash download_partition.sh
cd ..# TC tasks
conda activate fednlp
cd experiments/distributed/transformer_exps/run_tc_exps
python trial_error.py \
--dataset 20news\
--round -1 \
--depth 0 \
--width 8 \
--time_threshold 60 \
--max_round 3000 \The log files would be placed into experiments/distributed/transformer_exps/run_tc_exps/results/reproduce/20news-Trail-1-90. Tmp model would be saved into experiments/distributed/transformer_exps/run_tc_exps/tmp. The final accuracy results would be stored in experiments/distributed/transformer_exps/run_tc_exps/results/20news-depth-1-freq-90.log.
We process the result log via exps_data/draw-performance-baseline.ipynb
to get the final pictures in the manuscript.
Our experiment results could be downloaded from Google drive.
You can also refer to our artifact evaluation instructions in ae.pdf.