Skip to content

Commit 24760e7

Browse files
authored
Update README.md
1 parent bc59897 commit 24760e7

File tree

1 file changed

+9
-1
lines changed

1 file changed

+9
-1
lines changed

eval/scripts/README.md

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,18 +9,26 @@ ANTHROPIC_KEY = 'your_api_key'
99
GOOGLE_KEY = 'your_api_key' 
1010
```
1111

12-
For example, to create model results with `gpt-4o` and the default settings, go to the root of this repo and run
12+
For example, to create model results with `gpt-4o` and the default settings, go to the root of this repo and run
1313

1414
```bash
1515
python eval/scripts/gencode_json.py --model gpt-4o
1616
```
1717

18+
For results with scientist-annotated background, run
19+
20+
```bash
21+
python eval/scripts/gencode_json.py --model gpt-4o --with-background
22+
```
23+
24+
1825
### Command-Line Arguments
1926

2027
- `--model` - Specifies the model name used for generating responses.
2128
- `--output-dir` - Directory to store the generated code outputs (Default: `eval_results/generated_code`).
2229
- `--input-path` - Directory containing the JSON files describing the problems (Default: `eval/data/problems_all.jsonl`).
2330
- `--prompt-dir` - Directory where prompt files are saved (Default: `eval_results/prompt`).
31+
- `--with-background` - Include problem background if enabled.
2432
- `--temperature` - Controls the randomness of the generation (Default: 0).
2533

2634
## **Evaluate generated code**

0 commit comments

Comments
 (0)