Skip to content

Commit 9d7af54

Browse files
committed
add doc for result submission
1 parent 8645863 commit 9d7af54

File tree

1 file changed

+7
-1
lines changed

1 file changed

+7
-1
lines changed

README.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,8 @@
2121
<a href="#-quick-start">🔥 Quick Start</a> •
2222
<a href="#-remote-evaluation">🚀 Remote Evaluation</a> •
2323
<a href="#-llm-generated-code">💻 LLM-generated Code</a> •
24+
<a href="#-advanced-usage">🧑 Advanced Usage</a> •
25+
<a href="#-result-submission">📰 Result Submission</a> •
2426
<a href="#-citation">📜 Citation</a>
2527
</p>
2628

@@ -158,10 +160,14 @@ export GOOGLE_API_KEY=<your_google_api_key>
158160
We share pre-generated code samples from LLMs we have [evaluated](https://huggingface.co/spaces/bigcode/bigcodebench-leaderboard):
159161
* See the attachment of our [v0.2.0.post3](https://github.com/bigcode-project/bigcodebench/releases/tag/v0.2.0.post3). We include `sanitized_samples_calibrated.zip` for your convenience.
160162

161-
## Advanced Usage
163+
## 🧑 Advanced Usage
162164

163165
Please refer to the [ADVANCED USAGE](https://github.com/bigcode-project/bigcodebench/blob/main/ADVANCED_USAGE.md) for more details.
164166

167+
## 📰 Result Submission
168+
169+
Please email both the generated code samples and the execution results to [[email protected]](mailto:[email protected]) if you would like to contribute your model to the leaderboard. Note that the file names should be in the format of `[model_name]--[revision]--[bigcodebench|bigcodebench-hard]-[instruct|complete]--[backend]-[temp]-[n_samples]-sanitized_calibrated.jsonl` and `[model_name]--[revision]--[bigcodebench|bigcodebench-hard]-[instruct|complete]--[backend]-[temp]-[n_samples]-sanitized_calibrated_eval_results.json`. You can [file an issue](https://github.com/bigcode-project/bigcodebench/issues/new/choose) to remind us if we do not respond to your email within 3 days.
170+
165171
## 📜 Citation
166172

167173
```bibtex

0 commit comments

Comments
 (0)