For the environment, you can follow the instructions in the MiniGPT-5 official repository.
- Use the following commands to link CoMM dataset to the correct path:
ln -s ../../datasets/ ./datasets/CoMM-
Download Vicuna_7B and put it to
./checkpoints/Vicuna_7B -
Download prerained_minigpt4_7b.pth and put it to
./config/prerained_minigpt4_7b.pth -
Download stage1_cc3m.ckpt and put it to
./checkpoints/stage1_cc3m.ckpt -
Download our fine-tuned checkpoint and unzip it to
./checkpoints/MiniGPT-5
Use the following command to run inference:
bash eval.sh MiniGPT5 best_model val_data.pthbash train.sh