File tree Expand file tree Collapse file tree 4 files changed +12
-12
lines changed Expand file tree Collapse file tree 4 files changed +12
-12
lines changed Original file line number Diff line number Diff line change @@ -11,12 +11,12 @@ Few-Shot Learning 旨在研究如何从少量有监督的训练样本中学习
11
11
12
12
| 算法 | 预训练模型 | Score | eprstmt | bustm | ocnli | csldcp | tnews | cluewsc | iflytek | csl | chid |
13
13
| ------------ | ------------ | ------------ | ------------ | ------------ | ------------ | ------------ | ------------ | ------------ | ------------ | ------------ | ---------- |
14
- | P-tuning | ERNIE1 .0 | 55.70 | 83.28 | 63.43 | 35.36 | 60.54 | 50.02 | 54.51 | 50.14 | 54.93 | 41.16 |
15
- | P-tuning+Rdrop | ERNIE1 .0 | 56.17 | 83.11 | 64.50 | 35.48 | 61.88 | 57.51 | 54 | 52 | 56 | 41 |
16
- | EFL | ERNIE1 .0 | 54.47 | 84.10 | 60.10 | 35.12 | 56.61 | 56.57 | 53.59 | 46.37 | 61.21 | 36.56 |
17
- | EFL+Rdrop | ERNIE1 .0 | 56.94 | 87 | 62.75 | 37.54 | 53.98 | 56.77 | 56.87 | 48.54 | 62.19 | 46.85 |
18
- | PET | ERNIE1 .0 | 56.63 | 86.88 | 61.90 | 36.90 | 61.10 | 56.51 | 55.02 | 50.31 | 59.72 | 41.35 |
19
- | PET+Rdrop | ERNIE1 .0 | 57.74 | 87.54 | 61.57 | 37.30 | 62.5 | 58.91 | 56.25 | 53.46 | 61.84 | 40.31 |
14
+ | P-tuning | ERNIE-1 .0 | 55.70 | 83.28 | 63.43 | 35.36 | 60.54 | 50.02 | 54.51 | 50.14 | 54.93 | 41.16 |
15
+ | P-tuning+R-Drop | ERNIE-1 .0 | 56.23 | 83.11 | 64.56 | 35.71 | 61.88 | 57.51 | 54 | 52 | 56.3 | 41 |
16
+ | EFL | ERNIE-1 .0 | 54.47 | 84.10 | 60.10 | 35.12 | 56.61 | 56.57 | 53.59 | 46.37 | 61.21 | 36.56 |
17
+ | EFL+R-Drop | ERNIE-1 .0 | 56.94 | 87 | 62.75 | 37.54 | 53.98 | 56.77 | 56.87 | 48.54 | 62.19 | 46.85 |
18
+ | PET | ERNIE-1 .0 | 56.63 | 86.88 | 61.90 | 36.90 | 61.10 | 56.51 | 55.02 | 50.31 | 59.72 | 41.35 |
19
+ | PET+R-Drop | ERNIE-1 .0 | 57.37 | 87.54 | 63.66 | 36.46 | 62.5 | 58.91 | 56.25 | 53.46 | 57.22 | 40.31 |
20
20
21
21
## Models
22
22
- [ P-tuning] ( ./p-tuning )
Original file line number Diff line number Diff line change @@ -40,15 +40,15 @@ python -u -m paddle.distributed.launch --gpus "0" \
40
40
--learning_rate 5E-5 \
41
41
--epochs 10 \
42
42
--max_seq_length 512 \
43
- --rdrop_coef 0.5 \
43
+ --rdrop_coef 0 \
44
44
```
45
45
参数含义说明
46
46
- `task_name`: FewCLUE 中的数据集名字
47
47
- `negative_num`: 负样本采样个数,对于多分类任务,负样本数量对效果影响很大。负样本数量参数取值范围为 [1, class_num - 1]
48
48
- `device`: 使用 cpu/gpu 进行训练
49
49
- `save_dir`: 模型存储路径
50
50
- `max_seq_length`: 文本的最大截断长度
51
- - `rdrop_coef`: rdrop参数,默认为0,若为0则不使用rdrop
51
+ - `rdrop_coef`: R-Drop 策略 Loss 的权重系数,默认为 0, 若为 0 则未使用 R-Drop 策略
52
52
53
53
模型每训练 1 个 epoch, 会在验证集上进行评估,并针对测试集进行预测存储到预测结果文件。
54
54
Original file line number Diff line number Diff line change @@ -37,15 +37,15 @@ python -u -m paddle.distributed.launch --gpus "0" \
37
37
--learning_rate 5E-5 \
38
38
--epochs 10 \
39
39
--max_seq_length 512 \
40
- --rdrop_coef 0.5 \
40
+ --rdrop_coef 0 \
41
41
```
42
42
参数含义说明
43
43
- `task_name`: FewCLUE 中的数据集名字
44
44
- `p_embedding_num`: P-embedding 的个数
45
45
- `device`: 使用 cpu/gpu 进行训练
46
46
- `save_dir`: 模型存储路径
47
47
- `max_seq_length`: 文本的最大截断长度
48
- - `rdrop_coef`: rdrop参数,默认为0,若为0则不使用rdrop
48
+ - `rdrop_coef`: R-Drop 策略 Loss 的权重系数,默认为 0, 若为 0 则未使用 R-Drop 策略
49
49
50
50
模型每训练 1 个 epoch, 会在验证集和测试集上进行评估。
51
51
Original file line number Diff line number Diff line change @@ -39,15 +39,15 @@ python -u -m paddle.distributed.launch --gpus "0" \
39
39
--epochs 10 \
40
40
--max_seq_length 512 \
41
41
--language_model "ernie-1.0" \
42
- --rdrop_coef 0.5 \
42
+ --rdrop_coef 0 \
43
43
```
44
44
参数含义说明
45
45
- `task_name`: FewCLUE 中的数据集名字
46
46
- `device`: 使用 cpu/gpu 进行训练
47
47
- `pattern_id` 完形填空的模式
48
48
- `save_dir`: 模型存储路径
49
49
- `max_seq_length`: 文本的最大截断长度
50
- - `rdrop_coef`: rdrop参数,默认为0,若为0则不使用rdrop
50
+ - `rdrop_coef`: R-Drop 策略 Loss 的权重系数,默认为 0, 若为 0 则未使用 R-Drop 策略
51
51
52
52
模型每训练 1 个 epoch, 会在验证集上进行评估
53
53
You can’t perform that action at this time.
0 commit comments