|
| 1 | +--- |
| 2 | +description: In this section, we will provide a step-by-step guide to show how to annotate records in the UI. |
| 3 | +--- |
| 4 | +# Annotate your dataset |
| 5 | + |
| 6 | +!!! note |
| 7 | + To experience the UI features firsthand, you can take a look to the [Demo ↗](https://demo.argilla.io/sign-in?auth=ZGVtbzoxMjM0NTY3OA==). |
| 8 | + |
| 9 | +Argilla UI offers many functions to help you manage your annotation workflow, aiming to provide the most flexible approach to fit the wide variety of use cases handled by the community. |
| 10 | + |
| 11 | +## Annotation interface overview |
| 12 | + |
| 13 | +### Flexible layout |
| 14 | + |
| 15 | +=== "The header" |
| 16 | + |
| 17 | + At the right side of the navigation breadcrumb, you can customize the dataset settings and edit your profile. |
| 18 | + |
| 19 | +=== "The left pane" |
| 20 | + |
| 21 | + This area is displayed on top of **the control panel** for performing searches, applying filters and sorting results. The record card(s) are displayed one by one or in a vertical list depending on the active view: **Focus view** or **Bulk view.** A card includes one or many fields and an ellipsis menu to mainly access the record extra info like the metadata. |
| 22 | + |
| 23 | +=== "The right pane" |
| 24 | + |
| 25 | + This is where you annotate your dataset. Simply fill it out as a form, then choose to Submit, Save as Draft, or Discard your response to send the records to their corresponding queues. |
| 26 | + |
| 27 | +=== "The left bottom panel" |
| 28 | + |
| 29 | + This expandable area displays the annotation guidelines. |
| 30 | + |
| 31 | +=== "The right bottom panel" |
| 32 | + |
| 33 | + This expandable area displays your annotation progress. |
| 34 | + |
| 35 | + |
| 36 | + |
| 37 | +!!! tip |
| 38 | + The app is responsive which enable you to adapt your workspace from two to one column. You can even use your mobile to provides simple feedback on your datasets. |
| 39 | + |
| 40 | +### Shortcuts |
| 41 | + |
| 42 | +Argilla UI includes a range of shortcuts. For the main actions submit, discard, save as draft and the labels the keys are showed in the button. |
| 43 | + |
| 44 | +To move from one question to another or between records using the keyboard take a look at the table below. |
| 45 | + |
| 46 | +!!! tip |
| 47 | + Shortcuts provide a smoother experience, especially with a long list of labels or single-question forms. |
| 48 | + |
| 49 | +??? "Available shortcuts" |
| 50 | + |
| 51 | + | Action | Keys | |
| 52 | + | --- | --- | |
| 53 | + | Activate form | ⇥ Tab | |
| 54 | + | Move between questions | ↓ Down arrow or ↑ Up arrow | |
| 55 | + | Select and unselect label | 1, 2, 3 | |
| 56 | + | Move between labels or ranking options | ⇥ Tab or ⇧ Shift ⇥ Tab | |
| 57 | + | Select rating and rank | 1, 2, 3 | |
| 58 | + | Fit span to character selection | Hold ⇧ Shift | |
| 59 | + | Activate text area | ⇧ Shift ↵ Enter | |
| 60 | + | Exit text area | Esc | |
| 61 | + | Discard | ⌫ Backspace | |
| 62 | + | Save draft (Mac os) | ⌘ Cmd S | |
| 63 | + | Save draft (Other) | Ctrl S | |
| 64 | + | Submit | ↵ Enter | |
| 65 | + | Move between pages | → Right arrow or ← Left arrow | |
| 66 | + |
| 67 | +### View by status |
| 68 | + |
| 69 | +The view selector is set by default on Pending. |
| 70 | + |
| 71 | +If you are starting an annotation effort, all the records are initially kept in the Pending view. Once you start annotating, the records will move to the other queues: Draft, Submitted, Discarded. |
| 72 | + |
| 73 | +- **Pending**: The records without a response. |
| 74 | +- **Draft**: The records with partial responses. They can be submitted or discarded later. You can’t move them back to the pending queue. |
| 75 | +- **Discarded**: The records may or may not have responses. They can be edited but you can’t move them back to the pending queue. |
| 76 | +- **Submitted**: The records have been fully annotated and have already been submitted. |
| 77 | + |
| 78 | +### Suggestions |
| 79 | + |
| 80 | +If your dataset includes model predictions, you will see them represented by a sparkle icon `✨` in the label or value button. We call them “Suggestions” and they appear in the form as pre-filled responses. If you agree with the suggestion, you just need to click on the Submit button, and they will be considered as your response. |
| 81 | + |
| 82 | +If the suggestion is incorrect, you can modify it and submit your response. |
| 83 | + |
| 84 | +The score per suggested label/value is displayed for MultiLabelQuestion and RankingQuestion. |
| 85 | +For label questions, admins can also choose the ordering of labels in the datasets settings. Additionally, admins can choose to always show suggested labels at the beginning of the list. |
| 86 | + |
| 87 | +### Focus view |
| 88 | + |
| 89 | + |
| 90 | + |
| 91 | +This is the default view to annotate your dataset linearly, displaying one record after another. |
| 92 | + |
| 93 | +!!! tip |
| 94 | + You should use this when getting acquainted with a dataset or when the annotation team is very diverse, the topic is generic. |
| 95 | + |
| 96 | +Once you submit your first response, the next record will appear automatically. To see again your submitted record, just click on prev. |
| 97 | + |
| 98 | +**Navigating through the records** |
| 99 | + |
| 100 | +To navigate through the records, you can use the `Prev`, shown as `<`, and `Next`, `>` buttons on top of the record card. |
| 101 | + |
| 102 | +Each time the page is fully refreshed, the records with modified statuses (Pending to Discarded, Pending to Save as Draft, Pending to Submitted) are sent to the corresponding queue. The **control panel** displays the **status selector**, which is set to Pending by default. |
| 103 | + |
| 104 | +### Bulk view |
| 105 | + |
| 106 | + |
| 107 | + |
| 108 | +The bulk view is designed to speed up the annotation and get a quick overview of the whole dataset. |
| 109 | + |
| 110 | +It displays the records in a vertical list. Once the view is active, some functions from the **control panel** will be available to optimize the records reading. You define the number of records to display by page between `10`, `25`, `50` , `100` and the option to fix the card height by selecting `Expand records` or `Collapse records`. |
| 111 | + |
| 112 | +!!! tip |
| 113 | + You should use this when you have a good understanding of your data and want to apply your knowledge based on things like similarity search, filter patterns, and suggestion score thresholds. |
| 114 | + |
| 115 | +!!! note |
| 116 | + You should consider that bulk view does not show suggestions in bulk view (except for Spans) and will always convert questions to Draft queue when working with multiple question types. |
| 117 | + |
| 118 | +!!! tip |
| 119 | + With multiple questions, think about using the bulk view to annotate massively one question. Then, you can complete the annotation per records from the draft queue. |
| 120 | + |
| 121 | +### Annotation progress |
| 122 | + |
| 123 | +=== "General progress view" |
| 124 | + |
| 125 | + On the dataset list, the global progress of the annotation task from all users is displayed. This is indicated in the `Global progress` column, which shows the number of left records to be annotated, along with a progress bar. The progress bar displays the percentage and number of records submitted, conflicting (i.e., those with overlap), discarded, and pending by hovering your mouse over it. |
| 126 | + |
| 127 | +=== "Your own progress view" |
| 128 | + |
| 129 | + You can track your annotation progress in real-time. That means once you are annotating, the progress bar is incrementing in real time each time you submit or discard a record. Expanding the panel, the distribution of `Pending`, `Draft`, `Submitted` and `Discarded` responses is displayed in a donut chart. |
| 130 | + |
| 131 | +## Discover patterns and speed up your annotation. |
| 132 | + |
| 133 | +The UI offers various features designed to enhance your understanding of data patterns and streamline annotation tasks. Combining filters with bulk annotations can save you and your team hours of time. |
| 134 | + |
| 135 | +!!! tip |
| 136 | + You should use this when you are familiar with your data and have large volumes to annotate based on verified beliefs and experience. |
| 137 | + |
| 138 | +### Search and highlight |
| 139 | + |
| 140 | +From the **control panel** at the top of the left pane, you can search across the entire dataset or by fields (if you have more than one in your record) and visualize matched results highlighted in color. |
| 141 | + |
| 142 | +### Order by record semantic similarity |
| 143 | + |
| 144 | +You can retrieve records based on their similarity to another record if vectors have been added to the dataset. |
| 145 | + |
| 146 | +!!! note |
| 147 | + Consult these guides to know how to add vectors to your [dataset](dataset.md) and [records](record.md). |
| 148 | + |
| 149 | +To initiate a semantic search, click on `Find similar` within the record you wish to use as a reference. If multiple vectors are available, select the desired vector or choose whether to retrieve the most or least similar records. |
| 150 | + |
| 151 | +The retrieved records are then ordered by similarity, with the similarity score displayed on each record card. |
| 152 | + |
| 153 | +While the semantic search is active, you can update the selected vector or adjust the order of similarity, and specify the number of desired results. |
| 154 | + |
| 155 | +To cancel the search, click on the cross icon next to the reference record. |
| 156 | + |
| 157 | +### Filter and sort by metadata, responses, and suggestions |
| 158 | + |
| 159 | +<h4>Filter</h4> |
| 160 | + |
| 161 | +If the dataset contains metadata, responses and suggestions, click on **Filter** in the **control panel** to display the available filters. You can select multiple filters and combine them. |
| 162 | + |
| 163 | +!!! note |
| 164 | + Record info including metadata is visible from the ellipsis menu in the record card. |
| 165 | + |
| 166 | +=== "By metadata properties" |
| 167 | + |
| 168 | + From the `Metadata` dropdown, type and select the property. You can set a range for integer and float properties, and select specific values for term metadata. |
| 169 | + |
| 170 | +=== "By responses from the current user" |
| 171 | + |
| 172 | + From the `Responses` dropdown, type and select the question. You can set a range for rating questions and select specific values for label, multi-label, and span questions. |
| 173 | + |
| 174 | + !!! note |
| 175 | + The text and ranking questions are not available for filtering. |
| 176 | + |
| 177 | +=== "By suggestions" |
| 178 | + |
| 179 | + From the Suggestions dropdown, Filter the suggestions by `Suggestion values`, `Score` , or `Agent` |
| 180 | + |
| 181 | +<h4>Sort</h4> |
| 182 | + |
| 183 | +You can sort your records according to one or several attributes. |
| 184 | + |
| 185 | +The insertion time and last update are general to all records. |
| 186 | + |
| 187 | +The suggestion scores, response, and suggestion values for rating questions and metadata properties are available only when they were provided. |
| 188 | + |
| 189 | +## Annotate in teams |
| 190 | + |
| 191 | +!!! note |
| 192 | + Argilla 2.1 will come with automatic task distribution, which will allow you to distribute the work across several users more efficiently. |
| 193 | + |
| 194 | +### Edit guidelines in the settings |
| 195 | + |
| 196 | +As an `owner` or `admin`, you can edit the guidelines as much as you need from the icon settings on the header. Markdown format is enabled. |
| 197 | + |
| 198 | +!!! tip |
| 199 | + If you want further guidance on good practices for guidelines during the project development, check our [blog post](https://argilla.io/blog/annotation-guidelines-practices/). |
0 commit comments