You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In this example, you will learn how to apply the data module of Trinity-RFT to prepare the dataset before exploring and training. This example takes GSM-8K dataset as the example dataset to figure out:
5
+
In this example, you will learn how to apply the data processor workflow of Trinity-RFT to prepare and prioritize the dataset before task exploring and training. This example takes GSM-8K dataset as the example dataset to figure out:
6
6
7
-
1. how to prepare the data module
8
-
2. how to configure the data module
9
-
3. what the data module can do
7
+
1. how to prepare the data workflow
8
+
2. how to configure the data processor
9
+
3. what the data workflow can do
10
10
11
-
Before getting started, you need to prepare the main environment of Trinity-RFT according to the [installation section of the README file](../main.md), and you need to install [postgresql](https://www.postgresql.org/docs/current/tutorial-install.html) as well.
11
+
Before getting started, you need to prepare the main environment of Trinity-RFT according to the [installation section of the README file](../main.md).
12
12
13
13
### Data Preparation
14
14
15
-
#### Prepare the Data Module
15
+
#### Prepare the Data Workflow
16
16
17
-
As the overall framework of Trinity-RFT shows, the data module is one of the high-level functions. Trinity-RFT encapsulates the data module as an independent service to avoid dependency conflict issues. Thus you need to prepare a split environment for this module and start the server.
17
+
As the overall framework of Trinity-RFT shows, the data workflow is one of the high-level functions. Trinity-RFT encapsulates the data workflow as an independent service to avoid dependency conflict issues. Thus you need to prepare a split environment for this module and start the server.
18
18
19
19
```shell
20
-
# prepare split environments, including the one of data module
20
+
# prepare split environments, including the one of data workflow
21
21
python scripts/install.py
22
22
23
23
# start all split servers
24
24
python scripts/start_servers.py
25
25
```
26
26
27
-
### Configure the Data Module
27
+
### Configure the Data Processor
28
28
29
-
Trinity-RFT uses a unified config file to manage all config items. For the data module, you need to focus on the `data` section in the config file.
29
+
Trinity-RFT uses a unified config file to manage all config items. For the data processor, you need to focus on the `data_processor` section in the config file.
30
30
31
31
In this example, assume that you need to rank all math questions and corresponding answers by their difficulties. So you can set these config items like the following example:
32
32
33
33
```yaml
34
34
data_processor:
35
-
# basic info
36
-
source_data_path: '/path/to/gsm8k'
37
-
load_kwargs:
38
-
split: 'train'# only need the train split
39
-
format: # set the field mappings
40
-
prompt_key: 'question'
41
-
response_key: 'answer'
42
-
# database related. The result dataset will be stored in the database.
Here you can set the basic information for the GSM-8K dataset, database information that is used to store the result dataset, and some other items about downstream dataset loading for exploring and training:
54
+
Here you can set the basic buffers for the GSM-8K dataset input and output and some other items about downstream dataset loading for exploring and training:
47
55
48
-
+ `source_data_path`: the path to the raw dataset.
49
-
+ `load_kwargs`: extra config arguments for loading the raw dataset. Mainly for the `load_dataset` method in HuggingFace `datasets` library.
50
-
+ `format`: some dataset format config items, which are used to map original data field names to unified ones.
51
-
+ `db_url`: the URL of the postgresql database to store the result dataset.
56
+
+ `data_workflow_url`: the URL of the data processor service, which is started in the previous step.
57
+
+ `task_pipeline`: the configs for the task pipeline. Task pipeline is used to process the raw dataset. It consists of several inner configs:
58
+
+ `input_buffers`: the input buffers for the task pipeline. We usually load from raw dataset files in this pipeline, thus we need to the dataset `path` and set the `storage_type` to "file" and set `raw` to True. It allows multiple input buffers. We can name each buffer with the `name` field.
59
+
+ `output_buffer`: the output buffer for the task pipeline. We usually store the processed dataset in files as well, thus we need to set the `storage_type` to "file".
60
+
+ `format`: some dataset format config items, which are used to map original data field names to unified ones.
52
61
53
-
In addition, there are several config items related to the data active iterator, which is used to prepare a better dataset. The core part of the data active iterator, Data-Juicer, provides tens of operators to help clean or calculate key information for each sample in the dataset. You can configure this part depending on how familiar you are with Data-Juicer.
62
+
In addition, there are several config items related to the data active iterator in `task_pipeline` part, which is used to prepare a better dataset. The core part of the data active iterator, Data-Juicer, provides tens of operators to help clean or calculate key information for each sample in the dataset. You can configure this part depending on how familiar you are with Data-Juicer.
54
63
55
64
#### Not familiar with Data-Juicer
56
-
If you are not familiar with Data-Juicer, the data module provides a natural-language-based method to config the data processing recipe. What you need to do is only describe the demands of how you want to prepare for the raw dataset, and an agent will be invoked to arrange the data processing recipe for you. Here is an example:
65
+
If you are not familiar with Data-Juicer, the data processor provides a natural-language-based method to config the data processing recipe. What you need to do is only describe the demands of how you want to prepare for the raw dataset, and an agent will be invoked to arrange the data processing recipe for you. Here is an example:
57
66
58
67
```yaml
59
68
data_processor:
60
-
# basic info
61
-
source_data_path: '/path/to/gsm8k'
62
-
load_kwargs:
63
-
split: 'train' # only need the train split
64
-
format: # set the field mappings
65
-
prompt_key: 'question'
66
-
response_key: 'answer'
67
-
# database related. The result dataset will be stored in the database.
dj_process_desc: 'Please compute difficulty scores for these math questions.'
89
+
agent_model_name: 'qwen-max'
90
+
agent_model_config:
91
+
config_name: 'my-qwen-instruction'
92
+
model_type: 'dashscope_chat'
93
+
model_name: 'qwen2.5-72b-instruct'
94
+
clean_strategy: 'iterative'
78
95
```
79
96
80
97
You can write your demand description in config item `dj_process_desc`, and set the model name and configs used for the agent in config items `agent_model_name` and `agent_model_config`. Here we use Qwen2.5-72b-Instruct as our recipe managing agent. And you can set the `clean_strategy` to 'iterative' to get a better dataset.
@@ -99,19 +116,27 @@ After preparing the Data-Juicer data processing recipe, you can set the `dj_conf
99
116
100
117
```yaml
101
118
data_processor:
102
-
# basic info
103
-
source_data_path: '/path/to/gsm8k'
104
-
load_kwargs:
105
-
split: 'train' # only need the train split
106
-
format: # set the field mappings
107
-
prompt_key: 'question'
108
-
response_key: 'answer'
109
-
# database related. The result dataset will be stored in the database.
And you can set the `clean_strategy` to 'iterative' to get a better dataset.
@@ -123,7 +148,7 @@ All config items in the `data` section can be found [here](trinity_configs.md).
123
148
124
149
125
150
```{note}
126
-
Only when one of `dj_process_desc` and `dj_config_path` is provided, the data module and the data active iterator will be activated. Otherwise, this part will be skipped and it will enter into the exploring stage directly.
151
+
Only when one of `xxx_pipeline` is provided, and one of `dj_process_desc` and `dj_config_path` in the pipeline config is provided, the data workflow and the data active iterator will be activated. Otherwise, this part will be skipped and it will enter into the exploring stage directly.
127
152
```
128
153
129
154
### Exploring & Training
@@ -140,49 +165,60 @@ ray start --address=<master_address>
140
165
trinity run --config <Trinity-RFT_config_path>
141
166
```
142
167
143
-
If you follow the steps above, Trinity-RFT will send a request to the data module server, the data active iterator will be activated and compute difficulty scores for each sample in the raw dataset. After that, the data module server stores the result dataset into the database, when exploring begins, it will load the prepared dataset and continue the downstream steps.
168
+
If you follow the steps above, Trinity-RFT will send a request to the data workflow server, the data active iterator will be activated, compute difficulty scores for each sample in the raw dataset, and rank the dataset according to difficulty scores. After that, the data workflow server stores the result dataset into the output buffer, when exploring begins, it will load the prepared dataset and continue the downstream steps.
144
169
145
170
171
+
## Example: Data Processor for Experience Pipeline
146
172
147
-
## Example: human in the loop
173
+
TBD.
174
+
175
+
176
+
## Example: Human in the Loop
148
177
Sometimes, you might need to involve human feedbacks for some raw data. In this example, you will learn how to annotate raw data to get a better dataset before training. This example takes an example Q&A dataset and tries to select the chosen and rejected ones for DPO method.
149
178
150
-
Before getting started, you need to prepare the main environment of Trinity-RFT according to the installation section of the README file, install postgresql, and [start a label-studio server](https://github.com/modelscope/data-juicer/tree/main/tools/humanops) from Data-Juicer from source.
179
+
Before getting started, you need to prepare the main environment of Trinity-RFT according to the installation section of the README file, and [start a label-studio server](https://github.com/modelscope/data-juicer/tree/main/tools/humanops) from Data-Juicer from source.
151
180
152
181
### Data Preparation
153
182
154
-
#### Prepare the Data Module
183
+
#### Prepare the Data Workflow
155
184
156
-
As the overall framework of Trinity-RFT shows, the data module is one of the high-level functions. Trinity-RFT encapsulates the data module as an independent service to avoid dependency conflict issues. Thus you need to prepare a split environment for this module and start the server.
185
+
As the overall framework of Trinity-RFT shows, the data workflow is one of the high-level functions. Trinity-RFT encapsulates the data workflow as an independent service to avoid dependency conflict issues. Thus you need to prepare a split environment for this module and start the server.
157
186
158
187
```shell
159
-
# prepare split environments, including the one of data module
188
+
# prepare split environments, including the one of data workflow
160
189
python scripts/install.py
161
190
162
191
# start all split servers
163
192
python scripts/start_servers.py
164
193
```
165
194
166
-
### Configure the Data Module
195
+
### Configure the Data Workflow
167
196
168
-
Trinity-RFT uses a unified config file to manage all config items. For the data module, you need to focus on the `data` section in the config file.
197
+
Trinity-RFT uses a unified config file to manage all config items. For the data workflow, you need to focus on the `data_processor` section in the config file.
169
198
170
-
In this example, assume that you need to rank all math questions and corresponding answers by their difficulties. So you can set these config items like the following example:
199
+
In this example, assume that you need to select the chosen and rejected responses for DPO method. So you can set these config items like the following example:
Here you can set the basic information for the example dataset, database information that is used to store the result dataset, and some other items about downstream dataset loading for exploring and training, which is similar to the example above.
@@ -223,7 +259,7 @@ You can set more config items for this OP (e.g. notification when annotation is
223
259
224
260
### Start Running
225
261
226
-
When you start running with the RFT config, the data module will start the OP `human_preference_annotation_mapper`, and then you can find a new project on the "Projects" page of the label-studio server.
262
+
When you start running with the RFT config, the data workflow will start the OP `human_preference_annotation_mapper`, and then you can find a new project on the "Projects" page of the label-studio server.
0 commit comments