You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: 5411 docs update migrating to 20 flow 2 (#5430)
# Description
<!-- Please include a summary of the changes and the related issue.
Please also include relevant motivation and context. List any
dependencies that are required for this change. -->
Extended user and workspace merge v2.
Closes#5411
**Type of change**
<!-- Please delete options that are not relevant. Remember to title the
PR according to the type of change -->
- Documentation update
**How Has This Been Tested**
<!-- Please add some reference about how your feature has been tested.
-->
**Checklist**
<!-- Please go over the list and make sure you've taken everything into
account -->
- I added relevant documentation
- I followed the style guidelines of this project
- I did a self-review of my code
- I made corresponding changes to the documentation
- I confirm My changes generate no new warnings
- I have added tests that prove my fix is effective or that my feature
works
- I have added relevant notes to the CHANGELOG.md file (See
https://keepachangelog.com/)
---------
Co-authored-by: Paco Aranda <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Copy file name to clipboardExpand all lines: argilla/docs/how_to_guides/migrate_from_legacy_datasets.md
+88-19Lines changed: 88 additions & 19 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,21 +1,101 @@
1
-
# Migrate your legacy datasets to Argilla V2
1
+
# Migrate users, workspaces and datasets to Argilla 2.x
2
2
3
-
This guide will help you migrate task specific datasets to Argilla V2. These do not include the `FeedbackDataset` which is just an interim naming convention for the latest extensible dataset. Taskspecific datasets are datasets that are used for a specific task, such as text classification, token classification, etc. If you would like to learn about the backstory of SDK this migration, please refer to the [SDK migration blog post](https://argilla.io/blog/introducing-argilla-new-sdk/).
3
+
This guide will help you migrate task to Argilla V2. These do not include the `FeedbackDataset` which is just an interim naming convention for the latest extensible dataset. Task-specific datasets are datasets that are used for a specific task, such as text classification, token classification, etc. If you would like to learn about the backstory of SDK this migration, please refer to the [SDK migration blog post](https://argilla.io/blog/introducing-argilla-new-sdk/). Additionally, we will provide guidance on how to maintain your `User`'s and `Workspace`'s within the new Argilla V2 format.
4
4
5
5
!!! note
6
-
Legacy Datasets include: `DatasetForTextClassification`, `DatasetForTokenClassification`, and `DatasetForText2Text`.
6
+
Legacy datasets include: `DatasetForTextClassification`, `DatasetForTokenClassification`, and `DatasetForText2Text`.
7
+
8
+
`FeedbackDataset`'s do not need to be migrated as they are already in the Argilla V2 format. Anyway, since the 2.x version includes changes to the search index structure, you should reindex the datasets by enabling the docker environment variable REINDEX_DATASET (This step is automatically executed if you're running Argilla in an HF Space). See the [server configuration docs](../reference/argilla-server/configuration.md#docker-images-only) section for more details.
7
9
8
-
`FeedbackDataset`'s do not need to be migrated as they are already in the Argilla V2 format.
9
10
10
11
To follow this guide, you will need to have the following prerequisites:
11
12
12
13
- An argilla 1.* server instance running with legacy datasets.
13
14
- An argilla >=1.29 server instance running. If you don't have one, you can create one by following this [Argilla guide](../getting_started/quickstart.md).
14
15
- The `argilla` sdk package installed in your environment.
15
16
17
+
!!! warning
18
+
This guide will recreate all `User`'s' and `Workspace`'s' on a new server. Hence, they will be created with new passwords and IDs. If you want to keep the same passwords and IDs, you can can copy the datasets to a temporary v2 instance, then upgrade your current instance to v2.0 and copy the datasets back to your original instance after.
19
+
16
20
If your current legacy datasets are on a server with Argilla release after 1.29, you could chose to recreate your legacy datasets as new datasets on the same server. You could then upgrade the server to Argilla 2.0 and carry on working their. Your legacy datasets will not be visible on the new server, but they will remain in storage layers if you need to access them.
17
21
18
-
## Steps
22
+
For migrating the guides you will need to install the new `argilla` package. This includes a new `v1` module that allows you to connect to the Argilla V1 server.
23
+
24
+
```bash
25
+
pip install "argilla>=2.0.0"
26
+
```
27
+
28
+
## Migrate Users and Workspaces
29
+
30
+
The guide will take you through two steps:
31
+
32
+
1.**Retrieve the old users and workspaces** from the Argilla V1 server using the new `argilla` package.
33
+
2.**Recreate the users and workspaces** on the Argilla V2 server based op `name` as unique identifier.
34
+
35
+
### Step 1: Retrieve the old users and workspaces
36
+
37
+
You can use the `v1` module to connect to the Argilla V1 server.
38
+
39
+
```python
40
+
import argilla.v1 as rg_v1
41
+
42
+
# Initialize the API with an Argilla server less than 2.0
43
+
api_url ="<your-url>"
44
+
api_key ="<your-api-key>"
45
+
rg_v1.init(api_url, api_key)
46
+
```
47
+
48
+
Next, load the dataset `User` and `Workspaces` and from the Argilla V1 server:
49
+
50
+
```python
51
+
users_v1 = rg_v1.User.list()
52
+
workspaces_v1 = rg_v1.Workspace.list()
53
+
```
54
+
55
+
### Step 2: Recreate the users and workspaces
56
+
57
+
To recreate the users and workspaces on the Argilla V2 server, you can use the `argilla` package.
58
+
59
+
First, instantiate the `Argilla` class to connect to the Argilla V2 server:
60
+
61
+
```python
62
+
import argilla as rg
63
+
64
+
client = rg.Argilla()
65
+
```
66
+
67
+
Next, recreate the users and workspaces on the Argilla V2 server:
1. You need to chose a new password for the user, to do this programmatically you can use the `uuid` package to generate a random password. Take care to keep track of the passwords you chose, since you will not be able to retrieve them later.
95
+
96
+
Now you have successfully migrated your users and workspaces to Argilla V2 and can continue with the next steps.
97
+
98
+
## Migrate datasets
19
99
20
100
The guide will take you through three steps:
21
101
@@ -25,12 +105,7 @@ The guide will take you through three steps:
25
105
26
106
### Step 1: Retrieve the legacy dataset
27
107
28
-
Connect to the Argilla V1 server via the new `argilla` package. First, you should install an extra dependency:
29
-
```bash
30
-
pip install "argilla[legacy]"
31
-
```
32
-
33
-
Now, you can use the `v1` module to connect to the Argilla V1 server.
108
+
You can use the `v1` module to connect to the Argilla V1 server.
34
109
35
110
```python
36
111
import argilla.v1 as rg_v1
@@ -88,9 +163,7 @@ Next, define the new dataset settings:
88
163
```
89
164
90
165
1. The default field in `DatasetForTextClassification` is `text`, but make sure you provide all fields included in `record.inputs`.
91
-
92
166
2. Make sure you provide all relevant metadata fields available in the dataset.
93
-
94
167
3. Make sure you provide all relevant vectors available in the dataset.
95
168
96
169
=== "For multi-label classification"
@@ -113,9 +186,7 @@ Next, define the new dataset settings:
113
186
```
114
187
115
188
1. The default field in `DatasetForTextClassification` is `text`, but we should provide all fields included in `record.inputs`.
116
-
117
189
2. Make sure you provide all relevant metadata fields available in the dataset.
118
-
119
190
3. Make sure you provide all relevant vectors available in the dataset.
120
191
121
192
=== "For token classification"
@@ -138,7 +209,6 @@ Next, define the new dataset settings:
138
209
```
139
210
140
211
1. Make sure you provide all relevant metadata fields available in the dataset.
141
-
142
212
2. Make sure you provide all relevant vectors available in the dataset.
143
213
144
214
=== "For text generation"
@@ -161,21 +231,20 @@ Next, define the new dataset settings:
161
231
```
162
232
163
233
1. We should provide all relevant metadata fields available in the dataset.
164
-
165
234
2. We should provide all relevant vectors available in the dataset.
166
235
167
236
Finally, create the new dataset on the Argilla V2 server:
If a dataset with the same name already exists, the `create` method will raise an exception. You can check if the dataset exists and delete it before creating a new one.
0 commit comments