Skip to content

Commit bd08ef7

Browse files
committed
update README
1 parent e75900f commit bd08ef7

File tree

1 file changed

+19
-7
lines changed

1 file changed

+19
-7
lines changed

README.md

Lines changed: 19 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -98,18 +98,22 @@ As for the test samples, they can be found in `data/testing/test.csv`
9898

9999
```bash
100100
python3 training/variable_predictor.py \
101-
--training_csv_file data/training/train.csv
101+
--training_csv_file data/training/train.csv \
102+
--device <DEVICE>
102103
```
104+
If your machine has an NVIDIA GPU, set the `device` argument to `cuda`. If not specified, the default device will be `cpu`.
103105

104106
Where `training_csv_file` is the CSV file containg the training samples that was generated in Part 1.2. Alternatively, you could use the file that we have provided in the same section. The model's checkpoints will be saved in `training/var_runs`.
105107

106108
## Part 3: Evaluation
107109
1. Model evaluation is done through the `evaluation/model_eval.py`,
108110
```bash
109111
python3 evaluation/model_eval.py \
110-
--testing_csv_file data/testing/test.csv
112+
--testing_csv_file data/testing/test.csv \
111113
--checkpoint training/var_runs/<CHECKPOINT-NAME>
114+
--device <DEVICE>
112115
```
116+
Make sure to set `device` to `cuda` to run inference/evaluation on the GPU.
113117

114118
We also provide the model checkpoint hosted on [Huggingface](https://huggingface.co/spaces/scam2024/ReIdentify/tree/main?clone=true). To clone the repository, make sure to have `git lfs` installed on your machine. You can do so by following the official [documentation](https://git-lfs.com/).
115119
Once `git lfs` is properly installed, executing these commands,
@@ -122,16 +126,24 @@ git clone https://huggingface.co/spaces/scam2024/ReIdentify
122126
The checkpoint name is `model_26_2` and the evaluation can be executed as follow,
123127
```bash
124128
python3 evaluation/model_eval.py \
125-
--testing_csv_file data/testing/test.csv
129+
--testing_csv_file data/testing/test.csv \
126130
--checkpoint ReIdentify/model_26_2
127131
```
128132

129-
2. GPT4 evaluation: In the paper, we have run experiments to compare our model's performance in generating identifier names compared to GPT4 on the same test set mentioned in previous sections.
130-
Specifically, we used OpenAI's API service, hence, you will need an [OpenAI API key](https://platform.openai.com/docs/api-reference/authentication).
133+
2. GPT4 and Gemini Pro evaluation: In the paper, we have run experiments to compare our model's performance in generating identifier names compared to larger models, namely GPT4 and Gemini Pro, on the same test set mentioned in previous sections.
134+
Specifically, we used OpenAI's API service, hence, you will need an [OpenAI API key](https://platform.openai.com/docs/api-reference/authentication). Similarly, a Gemini API key is also required. It can be obtained through the following [link](https://ai.google.dev/gemini-api/docs/api-key).
131135

136+
To communicate with OpenAI's API,
132137
```bash
133-
python3 evaluation/ChatGPT_identifiers \
134-
--test_csv_file data/testing/test.csv
138+
python3 evaluation/ChatGPT_identifiers.py \
139+
--test_csv_file data/testing/test.csv \
140+
--api_key <API-KEY>
141+
```
142+
143+
To communicate with Gemini Pro's API,
144+
```bash
145+
python3 evaluation/Gemini_prompt.py \
146+
--test_csv_file data/testing/test.csv \
135147
--api_key <API-KEY>
136148
```
137149

0 commit comments

Comments
 (0)