Skip to content

Commit 7053e7f

Browse files
authored
update readme (#2)
* update readme * update data and model link * Update README.md
1 parent f86c9a1 commit 7053e7f

File tree

3 files changed

+16
-5
lines changed

3 files changed

+16
-5
lines changed

.DS_Store

6 KB
Binary file not shown.

README.md

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -15,13 +15,13 @@ pip install -r requirements.txt
1515

1616
## Datasets
1717

18-
Download the datasets from the following:
18+
Download the dataset from the following repository:
1919

2020
```
2121
https://github.com/lupantech/ScienceQA/tree/main/data
2222
```
2323

24-
Download the extracted vision fearures from [Anonymous](xxx) and unzip the files under `vision_features`
24+
Download the extracted vision features from [vision_features](https://drive.google.com/file/d/13B0hc_F_45-UlqPLKSgRz-ALtFQ8kIJr/view?usp=share_link) and unzip the files under `vision_features`
2525

2626
## Instructions
2727

@@ -47,7 +47,7 @@ CUDA_VISIBLE_DEVICES=0,1 python main.py \
4747

4848
### Inference
4949

50-
Our trained models are available at [Anonymous](xxx). To use our trained models, please put the them under the ```models``` folder.
50+
Our trained models are available at [models](https://drive.google.com/file/d/1FtTYOJPHnWnFfCxNC6M3gar4RAX5E21b/view?usp=share_link). To use our trained models, please put the them under the ```models``` folder.
5151

5252
```
5353
# rationale generation
@@ -56,7 +56,7 @@ CUDA_VISIBLE_DEVICES=0,1 python main.py \
5656
--user_msg rationale --img_type detr \
5757
--bs 8 --eval_bs 4 --eval_acc 10 --output_len 512 \
5858
--final_eval --prompt_format QCM-LE \
59-
--evaluate_dir models/rationale
59+
--evaluate_dir models/MM-CoT-UnifiedQA-base-Rationale
6060
6161
# answer inference
6262
CUDA_VISIBLE_DEVICES=0,1 python main.py \
@@ -66,7 +66,18 @@ CUDA_VISIBLE_DEVICES=0,1 python main.py \
6666
--final_eval --prompt_format QCMG-A \
6767
--eval_le models/rationale/predictions_ans_eval.json \
6868
--test_le models/rationale/predictions_ans_test.json \
69-
--evaluate_dir models/answer
69+
--evaluate_dir models/MM-CoT-UnifiedQA-base-Answer
70+
```
71+
72+
## Citing MM-CoT
73+
74+
```
75+
@article{zhang2023multicot,
76+
title={Multimodal Chain-of-Thought Reasoning in Language Models},
77+
author={Zhang, Zhuosheng and Zhang, Aston and Li, Mu and Zhao, Hai and Karypis, George and Smola, Alex},
78+
journal={arXiv preprint arXiv:2302.00923},
79+
year={2023}
80+
}
7081
```
7182

7283
## License

vision_features/mm-cot.png

-168 KB
Loading

0 commit comments

Comments
 (0)