Skip to content

Commit 3bbbbca

Browse files
shijinPJshijinPJ
committed
update: change to en
1 parent 255b659 commit 3bbbbca

File tree

2 files changed

+98
-97
lines changed

2 files changed

+98
-97
lines changed

‎docs/eval/dataset_redpajama.md‎

Lines changed: 43 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -1,57 +1,57 @@
11
# Dataset Redpajama
22

3-
## 数据集介绍
4-
本数据集旨在评估dingo内置提示词的准确性，因此选择开源数据集redpajama，从中抽取数据构建测试集。
3+
## Dataset Introduction
4+
This dataset aims to evaluate the accuracy of the built-in prompt words in dingo, therefore, the open-source dataset redpajama is selected, and data is extracted from it to build a test set.
55

6-
| 字段名 | 介绍 |
7-
|--------------|---------------------------|
8-
| data_id | 数据id，没有特殊含义，用户可根据自身需求修改 |
9-
| content | 待测试数据 |
10-
| language | 语言类型 |
11-
| error_status | 数据状态，True为负例数据，False为正例数据 |
12-
| type_list | 负例数据的负例类型，正例数据该字段则为空列表 |
13-
| name_list | 负例数据的负例名称，正例数据该字段则为空列表 |
14-
| reason_list | 负例数据的负例介绍，正例数据该字段则为空列表 |
6+
| Field Name | Description |
7+
|--------------|------------------------------------------------------------------------------------|
8+
| data_id | Data ID, without special meaning, users can modify it according to their own needs |
9+
| content | Data to be tested |
10+
| language | Language type |
11+
| error_status | Data status, True for negative examples, False for positive examples |
12+
| type_list | Negative types for negative examples, empty list for positive examples |
13+
| name_list | Negative names for negative examples, empty list for positive examples |
14+
| reason_list | Negative introductions for negative examples, empty list for positive examples |
1515

16-
链接：
17-
https://huggingface.co/datasets/chupei/redpajama_good_model
16+
Links:<br>
17+
https://huggingface.co/datasets/chupei/redpajama_good_model<br>
1818
https://huggingface.co/datasets/chupei/redpajama_bad_model
1919

20-
### 数据集构成
21-
| 类型 | 数量 |
22-
|---------------------------|-----|
23-
| 正例数据 | 101 |
24-
| 负例数据：disfluency | 4 |
25-
| 负例数据：dissimilarity | 3 |
26-
| 负例数据：disunderstandability | 2 |
27-
| 负例数据：incompleteness | 27 |
28-
| 负例数据：insecurity | 16 |
29-
| 负例数据：irrelevance | 49 |
20+
### Dataset Composition
21+
| Type | Count |
22+
|-----------------------------------------|-------|
23+
| Positive Examples | 101 |
24+
| Negative Examples: disfluency | 4 |
25+
| Negative Examples: dissimilarity | 3 |
26+
| Negative Examples: disunderstandability | 2 |
27+
| Negative Examples: incompleteness | 27 |
28+
| Negative Examples: insecurity | 16 |
29+
| Negative Examples: irrelevance | 49 |
3030

31-
## 提示词介绍
32-
本次测试使用内置的 **PromptTextQualityV2** 作为提示词，具体包含的内容可以参考：[PromptTextQualityV2介绍](../../dingo/model/prompt/prompt_text_quality_v2.py)
33-
内置的提示词集合可以参考：[提示词集合](../../dingo/model/prompt)
31+
## Prompt Introduction
32+
The built-in **PromptTextQualityV2** is used as the prompt for this test. Specific content can be referred to: [Introduction to PromptTextQualityV2](../../dingo/model/prompt/prompt_text_quality_v2.py)<br>
33+
The built-in prompt collection can be referred to: [Prompt Collection](../../dingo/model/prompt)
3434

35-
## 评测结果
36-
### 概念介绍
37-
正例数据与负例数据经过评测，均会生成对应的summary文件，因此需要对结果进行定义，明确概念。
35+
## Evaluation Results
36+
### Concept Introduction
37+
Both positive and negative examples will generate corresponding summary files after evaluation, so the results need to be defined and the concepts clarified.
3838

39-
| 名称 | 介绍 |
40-
|-----|-------------------------------|
41-
| TP | True Positive：正例数据中被评测为正例的数量 |
42-
| FP | False Positive：负例数据中被评测为正例的数量 |
43-
| TN | True Negative：负例数据中被评测为负例的数量 |
44-
| FN | False Negative：正例数据中被评测为负例的数量 |
45-
| 准确率 | TP / (TP + FP) 被评测为正例中正例数据的比率 |
46-
| 召回率 | TP / (TP + FN) 正例数据被评测为正例的比率 |
47-
| F1 | (准确率 + 召回率) / 2 |
39+
| Name | Description |
40+
|----------|-----------------------------------------------------------------------------|
41+
| TP | True Positive: Number of positive examples evaluated as positive |
42+
| FP | False Positive: Number of negative examples evaluated as positive |
43+
| TN | True Negative: Number of negative examples evaluated as negative |
44+
| FN | False Negative: Number of positive examples evaluated as negative |
45+
| Accuracy | TP / (TP + FP) Ratio of positive examples among those evaluated as positive |
46+
| Recall | TP / (TP + FN) Ratio of positive examples correctly evaluated as positive |
47+
| F1 | (Accuracy + Recall) / 2 |
4848

49-
### 结果展示
50-
| 数据集名称 | TP | FP | TN | FN | 准确率% | 召回率% | F1 |
51-
|-----------|----|----|-----|----|------|------|----|
52-
| redpajama | 95 | 0 | 101 | 6 | 100 | 94 | 97 |
49+
### Result Display
50+
| Dataset Name | TP | FP | TN | FN | Accuracy% | Recall% | F1 |
51+
|--------------|----|----|-----|----|-----------|---------|----|
52+
| redpajama | 95 | 0 | 101 | 6 | 100 | 94 | 97 |
5353

54-
## 评测方式
54+
## Evaluation Method
5555

5656
```python
5757
from dingo.io import InputArgs

‎docs/eval/dataset_slimpajama.md‎

Lines changed: 55 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -1,68 +1,69 @@
1-
# Dataset Slimpajama
1+
# Slimpajama Dataset
22

3-
## 数据集介绍
4-
本数据集旨在评估dingo内置规则的准确性，因此选择开源数据集slimpajama，从中抽取数据构建测试集。
3+
## Dataset Introduction
4+
This dataset aims to evaluate the accuracy of the built-in rules in dingo. Therefore, the open-source dataset Slimpajama was selected, and data was extracted from it to construct the test set.
55

6-
| 字段名 | 介绍 |
7-
|--------------|------------------------------------------|
8-
| data_id | 数据id，没有特殊含义，用户可根据自身需求修改 |
9-
| content | 待测试数据 |
10-
| language | 语言类型 |
11-
| error_status | 数据状态，True为负例数据，False为正例数据 |
12-
| type_list | 负例数据的负例类型，正例数据该字段则为空列表 |
13-
| name_list | 负例数据的负例名称，正例数据该字段则为空列表 |
14-
| reason_list | 负例数据的负例介绍，正例数据该字段则为空列表 |
6+
| Field Name | Description |
7+
|--------------|-------------------------------------------------------------------------------|
8+
| data_id | Data ID, without special meaning, can be modified according to user needs |
9+
| content | Data to be tested |
10+
| language | Language type |
11+
| error_status | Data status, True for negative examples, False for positive examples |
12+
| type_list | Negative example types for negative data, empty list for positive data |
13+
| name_list | Negative example names for negative data, empty list for positive data |
14+
| reason_list | Negative example descriptions for negative data, empty list for positive data |
1515

16-
链接：
16+
Links:
1717
https://huggingface.co/datasets/chupei/slimpajama_badcase_rule
1818
https://huggingface.co/datasets/chupei/slimpajama_goodcase_rule
1919

20-
### 数据集构成
21-
| 类型 | 数量 |
22-
|-----------------------------------|----|
23-
| 正例数据 | 82 |
24-
| 负例数据：RuleAlphaWords | 27 |
25-
| 负例数据：RuleCapitalWords | 26 |
26-
| 负例数据：RuleCharNumber | 5 |
27-
| 负例数据：RuleDocRepeat | 17 |
28-
| 负例数据：RuleHtmlEntity | 3 |
29-
| 负例数据：RuleLineEndWithEllipsis | 5 |
30-
| 负例数据：RuleLineEndWithTerminal | 5 |
31-
| 负例数据：RuleLineStartWithBulletpoint | 6 |
32-
| 负例数据：RuleLoremIpsum | 5 |
33-
| 负例数据：RuleMeanWordLength | 12 |
34-
| 负例数据：RuleNoPunc | 7 |
35-
| 负例数据：RuleSentenceNumber | 8 |
36-
| 负例数据：RuleSpecialCharacter | 4 |
37-
| 负例数据：RuleStopWord | 24 |
38-
| 负例数据：RuleSymbolWordRatio | 5 |
39-
| 负例数据：RuleUniqueWords | 7 |
40-
| 负例数据：RuleWordNumber | 7 |
20+
### Dataset Composition
21+
| Type | Count |
22+
|-------------------------------------------------|-------|
23+
| Positive examples | 82 |
24+
| Negative examples: RuleAlphaWords | 27 |
25+
| Negative examples: RuleCapitalWords | 26 |
26+
| Negative examples: RuleCharNumber | 5 |
27+
| Negative examples: RuleDocRepeat | 17 |
28+
| Negative examples: RuleHtmlEntity | 3 |
29+
| Negative examples: RuleLineEndWithEllipsis | 5 |
30+
| Negative examples: RuleLineEndWithTerminal | 5 |
31+
| Negative examples: RuleLineStartWithBulletpoint | 6 |
32+
| Negative examples: RuleLoremIpsum | 5 |
33+
| Negative examples: RuleMeanWordLength | 12 |
34+
| Negative examples: RuleNoPunc | 7 |
35+
| Negative examples: RuleSentenceNumber | 8 |
36+
| Negative examples: RuleSpecialCharacter | 4 |
37+
| Negative examples: RuleStopWord | 24 |
38+
| Negative examples: RuleSymbolWordRatio | 5 |
39+
| Negative examples: RuleUniqueWords | 7 |
40+
| Negative examples: RuleWordNumber | 7 |
4141

42-
## 规则介绍
43-
本次测试使用内置的 **pretrain** 作为eval_group，具体包含的规则可以参考：[集合介绍](../groups.md)
44-
集合内部的规则可以参考：[规则介绍](../rules.md)
42+
## Rules Introduction
43+
This test uses the built-in **pretrain** as the eval_group. For specific rules included, please refer to: [Group Introduction](../groups.md).<br>
44+
For rules within the group, please refer to: [Rules Introduction](../rules.md).
4545

46-
## 评测结果
47-
### 概念介绍
48-
正例数据与负例数据经过评测，均会生成对应的summary文件，因此需要对结果进行定义，明确概念。
46+
## Evaluation Results
47+
### Definitions
48+
After evaluation, both positive and negative data will generate corresponding summary files. Therefore, the results need to be defined with clear concepts.
4949

50-
| 名称 | 介绍 |
51-
|-----|-------------------------------|
52-
| TP | True Positive：正例数据中被评测为正例的数量 |
53-
| FP | False Positive：负例数据中被评测为正例的数量 |
54-
| TN | True Negative：负例数据中被评测为负例的数量 |
55-
| FN | False Negative：正例数据中被评测为负例的数量 |
56-
| 准确率 | TP / (TP + FP) 被评测为正例中正例数据的比率 |
57-
| 召回率 | TP / (TP + FN) 正例数据被评测为正例的比率 |
58-
| F1 | (准确率 + 召回率) / 2 |
50+
| Term | Description |
51+
|----------|--------------------------------------------------------------------------------|
52+
| TP | True Positive: Number of positive examples correctly identified |
53+
| FP | False Positive: Number of negative examples incorrectly identified as positive |
54+
| TN | True Negative: Number of negative examples correctly identified |
55+
| FN | False Negative: Number of positive examples incorrectly identified as negative |
56+
| Accuracy | TP / (TP + FP) Ratio of positive examples in the identified positives |
57+
| Recall | TP / (TP + FN) Ratio of positive examples correctly identified |
58+
| F1 | (Accuracy + Recall) / 2 |
5959

60-
### 结果展示
61-
| 数据集名称 | TP | FP | TN | FN | 准确率% | 召回率% | F1 |
62-
|------------|----|----|-----|----|------|------|------|
63-
| slimpajama | 78 | 5 | 103 | 4 | 94 | 95 | 94.5 |
60+
### Results Display
61+
| Dataset Name | TP | FP | TN | FN | Accuracy% | Recall% | F1 |
62+
|--------------|----|----|-----|----|-----------|---------|------|
63+
| slimpajama | 78 | 5 | 103 | 4 | 94 | 95 | 94.5 |
6464

65-
## 评测方式
65+
## Evaluation Method
66+
Translate this markdown into English.
6667

6768
```python
6869
from dingo.io import InputArgs

0 commit comments

Comments
 (0)