Skip to content

Commit 3082fc0

Browse files
LiuChiachiLiuChiachi
authored
Add ERNIE 3.0-Mini/Micro/Nano clue result (#2373)
* update ernie-3.0-tiny-clue-result * black acc value * update ernie3.0 readme * update format of table
1 parent e1014ee commit 3082fc0

File tree

2 files changed

+289
-49
lines changed

2 files changed

+289
-49
lines changed

‎examples/benchmark/clue/README.md‎

Lines changed: 167 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
<table style="width:100%;" cellpadding="2" cellspacing="0" border="1" bordercolor="#000000">
1313
<tbody>
1414
<tr>
15-
<td style="text-align:center">
15+
<td style="text-align:center;vertical-align:middle">
1616
<span style="font-size:18px;">Arch</span>
1717
</td>
1818
<td style="text-align:center">
@@ -321,42 +321,42 @@
321321
</td>
322322
</tr>
323323
<tr>
324-
<td rowspan=5 align=center> 6L768H </td>
324+
<td rowspan=5 align=center> 6L768H </td>
325325
<td style="text-align:center">
326326
<span style="font-size:18px">ERNIE 3.0-Medium-zh</span>
327327
</td>
328328
<td style="text-align:center">
329-
<span style="font-size:18px"><b>72.49</b></span>
329+
<span style="font-size:18px"><b>72.49</b></span>
330330
</td>
331331
<td style="text-align:center">
332-
<span style="font-size:18px"><b>73.37</b></span>
332+
<span style="font-size:18px"><b>73.37</b></span>
333333
</td>
334334
<td style="text-align:center">
335-
<span style="font-size:18px"><b>57.00</b></span>
335+
<span style="font-size:18px"><b>57.00</b></span>
336336
</td>
337337
<td style="text-align:center">
338-
<span style="font-size:18px"><b>60.67</b></span>
338+
<span style="font-size:18px"><b>60.67</b></span>
339339
</td>
340340
<td style="text-align:center">
341-
<span style="font-size:18px"><b>80.64</b></span>
341+
<span style="font-size:18px"><b>80.64</b></span>
342342
</td>
343343
<td style="text-align:center">
344-
<span style="font-size:18px"><b>76.88</b></span>
344+
<span style="font-size:18px"><b>76.88</b></span>
345345
</td>
346346
<td style="text-align:center">
347-
<span style="font-size:18px"><b>79.28</b></span>
347+
<span style="font-size:18px"><b>79.28</b></span>
348348
</td>
349349
<td style="text-align:center">
350-
<span style="font-size:18px"><b>81.60</b></span>
350+
<span style="font-size:18px"><b>81.60</b></span>
351351
</td>
352352
<td style="text-align:center">
353-
<span style="font-size:18px"><b>65.83/87.30</b></span>
353+
<span style="font-size:18px"><b>65.83/87.30</b></span>
354354
</td>
355355
<td style="text-align:center">
356-
<span style="font-size:18px"><b>79.91</b></span>
356+
<span style="font-size:18px"><b>79.91</b></span>
357357
</td>
358358
<td style="text-align:center">
359-
<span style="font-size:18px"><b>69.73</b></span>
359+
<span style="font-size:18px"><b>69.73</b></span>
360360
</td>
361361
</tr>
362362
<tr>
@@ -402,112 +402,229 @@
402402
<span style="font-size:18px">TinyBERT<sub>6</sub>, Chinese</span>
403403
</td>
404404
<td style="text-align:center">
405-
<span style="font-size:18px">69.58</span>
405+
<span style="font-size:18px">69.58</span>
406406
</td>
407407
<td style="text-align:center">
408-
<span style="font-size:18px">72.22</span>
408+
<span style="font-size:18px">72.22</span>
409409
</td>
410410
<td style="text-align:center">
411-
<span style="font-size:18px">55.70</span>
411+
<span style="font-size:18px">55.70</span>
412412
</td>
413413
<td style="text-align:center">
414-
<span style="font-size:18px">54.48</span>
414+
<span style="font-size:18px">54.48</span>
415415
</td>
416416
<td style="text-align:center">
417-
<span style="font-size:18px">79.12</span>
417+
<span style="font-size:18px">79.12</span>
418418
</td>
419419
<td style="text-align:center">
420-
<span style="font-size:18px">74.07</span>
420+
<span style="font-size:18px">74.07</span>
421421
</td>
422422
<td style="text-align:center">
423-
<span style="font-size:18px">77.63</span>
423+
<span style="font-size:18px">77.63</span>
424424
</td>
425425
<td style="text-align:center">
426-
<span style="font-size:18px">80.17</span>
426+
<span style="font-size:18px">80.17</span>
427427
</td>
428428
<td style="text-align:center">
429-
<span style="font-size:18px">62.63/83.72</span>
429+
<span style="font-size:18px">62.63/83.72</span>
430430
</td>
431431
<td style="text-align:center">
432-
<span style="font-size:18px">77.64</span>
432+
<span style="font-size:18px">77.64</span>
433433
</td>
434434
<td style="text-align:center">
435-
<span style="font-size:18px">62.11</span>
435+
<span style="font-size:18px">62.11</span>
436436
</td>
437437
</tr>
438438
<td style="text-align:center">
439439
<span style="font-size:18px">RoFormerV2 Small</span>
440440
</td>
441441
<td style="text-align:center">
442-
<span style="font-size:18px">68.52</span>
442+
<span style="font-size:18px">68.52</span>
443443
</td>
444444
<td style="text-align:center">
445-
<span style="font-size:18px">72.47</span>
445+
<span style="font-size:18px">72.47</span>
446446
</td>
447447
<td style="text-align:center">
448-
<span style="font-size:18px">56.53</span>
448+
<span style="font-size:18px">56.53</span>
449449
</td>
450450
<td style="text-align:center">
451-
<span style="font-size:18px">60.72</span>
451+
<span style="font-size:18px">60.72</span>
452452
</td>
453453
<td style="text-align:center">
454-
<span style="font-size:18px">76.37</span>
454+
<span style="font-size:18px">76.37</span>
455455
</td>
456456
<td style="text-align:center">
457-
<span style="font-size:18px">72.95</span>
457+
<span style="font-size:18px">72.95</span>
458458
</td>
459459
<td style="text-align:center">
460-
<span style="font-size:18px">75.00</span>
460+
<span style="font-size:18px">75.00</span>
461461
</td>
462462
<td style="text-align:center">
463-
<span style="font-size:18px">81.07</span>
463+
<span style="font-size:18px">81.07</span>
464464
</td>
465465
<td style="text-align:center">
466-
<span style="font-size:18px">62.97/83.64</span>
466+
<span style="font-size:18px">62.97/83.64</span>
467467
</td>
468468
<td style="text-align:center">
469-
<span style="font-size:18px">67.66</span>
469+
<span style="font-size:18px">67.66</span>
470470
</td>
471471
<td style="text-align:center">
472-
<span style="font-size:18px">59.41</span>
472+
<span style="font-size:18px">59.41</span>
473473
</td>
474474
</tr>
475475
<tr>
476476
<td style="text-align:center">
477477
<span style="font-size:18px">UER/Chinese-RoBERTa (L6-H768)</span>
478478
</td>
479479
<td style="text-align:center">
480-
<span style="font-size:18px">66.67</span>
480+
<span style="font-size:18px">66.67</span>
481+
</td>
482+
<td style="text-align:center">
483+
<span style="font-size:18px">70.13</span>
484+
</td>
485+
<td style="text-align:center">
486+
<span style="font-size:18px">56.41</span>
487+
</td>
488+
<td style="text-align:center">
489+
<span style="font-size:18px">59.79</span>
490+
</td>
491+
<td style="text-align:center">
492+
<span style="font-size:18px">77.38</span>
493+
</td>
494+
<td style="text-align:center">
495+
<span style="font-size:18px">71.86</span>
496+
</td>
497+
<td style="text-align:center">
498+
<span style="font-size:18px">69.41</span>
499+
</td>
500+
<td style="text-align:center">
501+
<span style="font-size:18px">76.73</span>
502+
</td>
503+
<td style="text-align:center">
504+
<span style="font-size:18px">53.22/75.03</span>
505+
</td>
506+
<td style="text-align:center">
507+
<span style="font-size:18px">77.00</span>
508+
</td>
509+
<td style="text-align:center">
510+
<span style="font-size:18px">54.77</span>
511+
</td>
512+
</tr>
513+
<tr>
514+
<td rowspan=1 align=center> 6L384H </td>
515+
<td style="text-align:center">
516+
<span style="font-size:18px">ERNIE 3.0-Mini-zh</span>
517+
</td>
518+
<td style="text-align:center">
519+
<span style="font-size:18px"><b>66.90</b></span>
520+
</td>
521+
<td style="text-align:center">
522+
<span style="font-size:18px"><b>71.85</b></span>
523+
</td>
524+
<td style="text-align:center">
525+
<span style="font-size:18px"><b>55.24</b></span>
526+
</td>
527+
<td style="text-align:center">
528+
<span style="font-size:18px"><b>54.48</b></span>
529+
</td>
530+
<td style="text-align:center">
531+
<span style="font-size:18px"><b>77.19</b></span>
532+
</td>
533+
<td style="text-align:center">
534+
<span style="font-size:18px"><b>73.08</b></span>
535+
</td>
536+
<td style="text-align:center">
537+
<span style="font-size:18px"><b>71.05</b></span>
481538
</td>
482539
<td style="text-align:center">
483-
<span style="font-size:18px">70.13</span>
540+
<span style="font-size:18px"><b>79.30</b></span>
484541
</td>
485542
<td style="text-align:center">
486-
<span style="font-size:18px">56.41</span>
543+
<span style="font-size:18px"><b>58.53/81.97</b></span>
487544
</td>
488545
<td style="text-align:center">
489-
<span style="font-size:18px">59.79</span>
546+
<span style="font-size:18px"><b>69.71</b></span>
547+
</td>
548+
<td style="text-align:center">
549+
<span style="font-size:18px"><b>58.60</b></span>
550+
</td>
551+
</tr>
552+
<tr>
553+
<td rowspan=1 align=center> 4L384H </td>
554+
<td style="text-align:center">
555+
<span style="font-size:18px">ERNIE 3.0-Micro-zh</span>
556+
</td>
557+
<td style="text-align:center">
558+
<span style="font-size:18px"><b>64.21</b></span>
559+
</td>
560+
<td style="text-align:center">
561+
<span style="font-size:18px"><b>71.15</b></span>
562+
</td>
563+
<td style="text-align:center">
564+
<span style="font-size:18px"><b>55.05</b></span>
565+
</td>
566+
<td style="text-align:center">
567+
<span style="font-size:18px"><b>53.83</b></span>
568+
</td>
569+
<td style="text-align:center">
570+
<span style="font-size:18px"><b>74.81</b></span>
571+
</td>
572+
<td style="text-align:center">
573+
<span style="font-size:18px"><b>70.41</b></span>
574+
</td>
575+
<td style="text-align:center">
576+
<span style="font-size:18px"><b>69.08</b></span>
490577
</td>
491578
<td style="text-align:center">
492-
<span style="font-size:18px">77.38</span>
579+
<span style="font-size:18px"><b>76.50</b></span>
493580
</td>
494581
<td style="text-align:center">
495-
<span style="font-size:18px">71.86</span>
582+
<span style="font-size:18px"><b>53.77/77.82</b></span>
496583
</td>
497584
<td style="text-align:center">
498-
<span style="font-size:18px">69.41</span>
585+
<span style="font-size:18px"><b>62.26</b></span>
586+
</td>
587+
<td style="text-align:center">
588+
<span style="font-size:18px"><b>55.53</b></span>
589+
</td>
590+
</tr>
591+
<tr>
592+
<td rowspan=1 align=center> 4L312H </td>
593+
<td style="text-align:center">
594+
<span style="font-size:18px">ERNIE 3.0-Nano-zh</span>
595+
</td>
596+
<td style="text-align:center">
597+
<span style="font-size:18px"><b>62.97</b></span>
598+
</td>
599+
<td style="text-align:center">
600+
<span style="font-size:18px"><b>70.51</b></span>
499601
</td>
500602
<td style="text-align:center">
501-
<span style="font-size:18px">76.73</span>
603+
<span style="font-size:18px"><b>54.57</b></span>
502604
</td>
503605
<td style="text-align:center">
504-
<span style="font-size:18px">53.22/75.03</span>
606+
<span style="font-size:18px"><b>48.36</b></span>
505607
</td>
506608
<td style="text-align:center">
507-
<span style="font-size:18px">77.00</span>
609+
<span style="font-size:18px"><b>74.97</b></span>
508610
</td>
509611
<td style="text-align:center">
510-
<span style="font-size:18px">54.77</span>
612+
<span style="font-size:18px"><b>70.61</b></span>
613+
</td>
614+
<td style="text-align:center">
615+
<span style="font-size:18px"><b>68.75</b></span>
616+
</td>
617+
<td style="text-align:center">
618+
<span style="font-size:18px"><b>75.93</b></span>
619+
</td>
620+
<td style="text-align:center">
621+
<span style="font-size:18px"><b>52.00/76.35</b></span>
622+
</td>
623+
<td style="text-align:center">
624+
<span style="font-size:18px"><b>58.91</b></span>
625+
</td>
626+
<td style="text-align:center">
627+
<span style="font-size:18px"><b>55.11</b></span>
511628
</td>
512629
</tr>
513630
<tbody>
@@ -551,9 +668,12 @@ AFQMC、TNEWS、IFLYTEK、CMNLI、OCNLI、CLUEWSC2020、CSL 、CHID 和 C<sup>3<
551668
| RoFormerV2 Small | 5e-5,16 | 2e-5,16 | 5e-5,16 | 5e-5,32 | 2e-5,16 | 3e-5,8 | 3e-5,16 | 3e-5,24 | 3e-5,24 | 3e-5,24 |
552669
| HLF/RBT6, Chinese | 3e-5,16 | 5e-5,16 | 5e-5,16 | 5e-5,64 | 3e-5,32 | 3e-5,32 | 3e-5,16 | 3e-5,32 | 3e-5,24 | 3e-5,24 |
553670
| UER/Chinese-RoBERTa (L6-H768) | 2e-5,16 | 5e-5,32 | 5e-5,16 | 5e-5,32 | 3e-5,16 | 5e-5,8 | 3e-5,16 | 3e-5,24 | 3e-5,24 | 3e-5,32 |
671+
| ERNIE 3.0-Mini-zh | 5e-5,64 | 5e-5,64 | 5e-5,16 | 5e-5,32 | 2e-5,16 | 2e-5,8 | 2e-5,16 | 3e-5,24 | 3e-5,24 | 3e-5,24 |
672+
| ERNIE 3.0-Micro-zh | 3e-5,16 | 5e-5,32 | 5e-5,16 | 5e-5,16 | 2e-5,32 | 5e-5,16 | 3e-5,64 | 3e-5,24 | 3e-5,32 | 3e-5,24 |
673+
| ERNIE 3.0-Nano-zh | 2e-5,32 | 5e-5,16 | 5e-5,16 | 5e-5,16 | 3e-5,16 | 1e-5,8 | 3e-5,32 | 3e-5,24 | 3e-5,24 | 2e-5,24 |
554674

555675

556-
其中，`ERNIE 3.0-Base-zh`、`ERNIE 3.0-Medium-zh`、`ERNIE-Gram-zh`、`ERNIE 1.0` 在 CLUEWSC2020 处的 dropout_prob 为 0.0，`ERNIE 3.0-Base-zh`、`HLF/RBT6, Chinese`、`Mengzi-BERT-Base`、`ERNIE-Gram-zh`、`ERNIE 1.0` 、`TinyBERT6, Chinese`、`UER/Chinese-RoBERTa (L6-H768)` 在 IFLYTEK 处的 dropout_prob 为 0.0。
676+
其中，`ERNIE 3.0-Base-zh`、`ERNIE 3.0-Medium-zh`、`ERNIE-Gram-zh`、`ERNIE 1.0`、`ERNIE 3.0-Mini-zh`、`ERNIE 3.0-Micro-zh`、`ERNIE 3.0-Nano-zh` 在 CLUEWSC2020 处的 dropout_prob 为 0.0，`ERNIE 3.0-Base-zh`、`HLF/RBT6, Chinese`、`Mengzi-BERT-Base`、`ERNIE-Gram-zh`、`ERNIE 1.0` 、`TinyBERT6, Chinese`、`UER/Chinese-RoBERTa (L6-H768)`、`ERNIE 3.0-Mini-zh`、`ERNIE 3.0-Micro-zh`、`ERNIE 3.0-Nano-zh` 在 IFLYTEK 处的 dropout_prob 为 0.0。
557677

558678

559679
## 一键复现模型效果

0 commit comments

Comments
 (0)