|
18 | 18 | <link href=" https://cdn.jsdelivr.net/npm/[email protected]/themes/prism.css" rel=" stylesheet" /> |
19 | 19 | <script src=" https://cdn.jsdelivr.net/npm/[email protected]/components/prism-core.min.js" ></script> |
20 | 20 | <script src=" https://cdn.jsdelivr.net/npm/[email protected]/plugins/autoloader/prism-autoloader.min.js" ></script> |
| 21 | + <script src=" https://cdn.jsdelivr.net/npm/[email protected]/prism-bibtex.min.js" ></script> |
21 | 22 |
|
22 | 23 | <style> |
23 | 24 | body { |
@@ -63,9 +64,9 @@ <h1 class="text-nowrap mt-5" style="font-size: xx-large;"> |
63 | 64 | <div> |
64 | 65 | 🚀 Code Efficiency Evaluation requires: |
65 | 66 | <ul> |
66 | | - <li><strong>Performance-exercising tasks & inputs</strong> |
| 67 | + <li><strong>Performance-exercising tasks & inputs --</strong> "all complexities are equal when N is small" |
67 | 68 | </li> |
68 | | - <li><strong>Meaningful compound metric:</strong> |
| 69 | + <li><strong>Meaningful compound metric --</strong> avg. speedup does not fit multi-task evaluation |
69 | 70 | </li> |
70 | 71 | </ul> |
71 | 72 | <p>Based on <strong>Differential Performance Evaluation</strong>, the EvalPerf dataset (current |
@@ -96,6 +97,17 @@ <h1 class="text-nowrap mt-5" style="font-size: xx-large;"> |
96 | 97 | <table id="leaderboard" |
97 | 98 | class="table table-responsive table-striped table-bordered flex-shrink-1 border border-5"> |
98 | 99 | </table> |
| 100 | + <h2 id="sponsor" class="text-nowrap mt-5">🖊️ Citation</h2> |
| 101 | + <pre style="padding-top: 0; padding-bottom: 0;"> |
| 102 | + <code class="language-bibtex"> |
| 103 | +@inproceedings{evalperf, |
| 104 | + title = {Evaluating Language Models for Efficient Code Generation}, |
| 105 | + author = {Liu, Jiawei and Xie, Songrun and Wang, Junhao and Wei, Yuxiang and Ding, Yifeng and Zhang, Lingming}, |
| 106 | + booktitle = {First Conference on Language Modeling}, |
| 107 | + year = {2024}, |
| 108 | + url = {https://openreview.net/forum?id=IBCBMeAhmC}, |
| 109 | +}</code> |
| 110 | + </pre> |
99 | 111 | <h2 id="sponsor" class="text-nowrap mt-5">🤗 Acknowledgment</h2> |
100 | 112 | <p> |
101 | 113 | We thank |
|
0 commit comments