You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -103,7 +103,7 @@ bigcodebench.generate \
103
103
The generated code samples will be stored in a file named `[model_name]--bigcodebench-[nl2c|c2c]--[backend]-[temp]-[n_samples].jsonl`. Alternatively, you can use the following command to utilize our pre-built docker images for generating code samples:
# Sanitized code will be produced to `/path/to/vicuna-[??]b_temp_[??]-sanitized`
161
161
```
162
162
163
-
<details><summary>🔎 Checking the compilability of post-processed code<i>:: click to expand ::</i></summary>
163
+
<details><summary>🔎 Checking the compatibility of post-processed code<i>:: click to expand ::</i></summary>
164
164
<div>
165
165
166
166
To double-check the post-processing results, you can use `bigcodebench.syncheck` to check the code validity before and after sanitization, which will print erroneous code snippets and why they are wrong:
@@ -183,9 +183,9 @@ You are strongly recommended to use a sandbox such as [docker](https://docs.dock
183
183
184
184
```shell
185
185
# mount the current directory to the container
186
-
docker run -v $(pwd):/bigcodebench codeeval/code-eval-evaluate:latest --dataset bigcodebench --samples samples.jsonl
186
+
docker run -v $(pwd):/bigcodebench terryzho/bigcodebench-evaluate:latest --subset [c2c|nl2c] --samples samples.jsonl
0 commit comments