fix: FIT-1172: Anno Eval: Onboarding option is active even if 'manual task assignment' option is selected (#9124)

Co-authored-by: yyassi-heartex <yyassi-heartex@users.noreply.github.com>

Author: yyassi-heartex

docs/source/guide/admin_settings.md

@@ -16,6 +16,6 @@ There are several places where you can configure org-wide settings:
 | ------------- | ------------ | ------------ |
 | [**Usage & License**](admin_usage) | Owner<br />Admin | This page has a mix of settings that can be set by Administrators (enabling email notifications) and Owners (enabling AI, enabling storage proxy, enabling early adopter features).   | 
 | [**Access Token**](access_tokens) | Owner<br />Admin | Control which types of access tokens are available in the organization.  | 
-| [**Model Providers**](model_providers)         | Owner<br />Admin | Set up model providers that can be used with [Prompts](prompts_overview) and [Chat](/tags/chat). | 
+| [**Model Providers**](model_providers)         | Owner<br />Admin | Set up model providers that can be used with [Prompts](prompts_overview) and [Chat](/tags/chat.html). | 
 | [**Permissions**](admin_permissions)        |  Owner | Customize certain permissions for roles.  | 
 | [**Support reports**](support_reports)      | Owner<br />Admin | Generate anonymized operational reports that help HumanSignal support understand your deployment, diagnose issues, and recommend workflow and performance improvements. |

docs/source/guide/model_providers.md

@@ -15,7 +15,7 @@ date: 2025-02-18 12:03:59
 
 To use certain AI features across your organization, you must first set up a model provider. You can set up model providers from **Organization > Settings**.
 
-For example, if you want to interact with an LLM when using the [`<Chat>` tag](/tags/chat), you will first need to configure access to the model.
+For example, if you want to interact with an LLM when using the [`<Chat>` tag](/tags/chat.html), you will first need to configure access to the model.
 
 
 !!! note

docs/source/guide/project_settings_lse.md

@@ -559,7 +559,7 @@ By default, each task only needs to be annotated by one annotator. If you want m
 
 The number of distinct annotations you want to allow per task. 
 
-Note that in certain situations, this may be exceeded. For example, if there are long-standing drafts within a project or you have a very low [task reservation](#task-lock) time. 
+Note that in certain situations, this may be exceeded. For example, if there are long-standing drafts within a project or you have a very low [task reservation](#lock-tasks) time. 
 
 Also note that only annotations created by distinct users count towards the overlap. For example, if the overlap is `2` and a user creates and submits two annotations on a single task (which can be done in Quick View), the overlap threshold will not be reached until another user submits an annotation. 
 
@@ -642,7 +642,12 @@ For more information about pausing annotators, including how to manually pause s
 
 <dd>
 
-Evaluate annotators against [ground truths](ground_truths) within a project. A “ground truth” annotation is a verified, high-quality annotation that serves as the correct answer for a specific task.
+!!! note
+    Annotator Evaluation settings are only available when the project is configured to [automatically assign tasks](#distribute-tasks). If you are using Manual distribution, this section will not appear in your project settings.
+    
+    If you switch a project from Automatic to Manual distribution, annotator evaluation is automatically disabled.
+
+Evaluate annotators against [ground truths](ground_truths) within a project. A "ground truth" annotation is a verified, high-quality annotation that serves as the correct answer for a specific task.
 
 When enabled, this setting looks at the agreement score for the annotator when compared solely against ground truth annotations. You can decide to automatically pause an annotator within the project if their ground truth agreement score falls below a certain threshold. 
 
@@ -681,7 +686,7 @@ When annotators enter the labeling stream, they are first presented with tasks t
 
 Use the counter to determine how many ground truth tasks should be presented first before the annotator progresses through the remaining project tasks. 
 
-**Note:** This option is only active when the project is configured to [automatically assign tasks](#distribute-tasks). If you are using Manual distribution, annotators will see tasks ordered by ID number. If you would like them to see ground truth tasks first, you should add ground truth annotations in the same order. 
+Set this counter to zero if you want to skip onboarding and only use continuous evaluation. 
 </td>
 </tr>
 <tr>

docs/source/guide/release_notes/onprem/2.29.0.md

@@ -18,7 +18,7 @@ Chat conversations are now a native data type in Label Studio, so you can annota
 
 For more information, see:
 
-[**Chat tag page**](/tags/chat)
+[**Chat tag page**](/tags/chat.html)
 
 [**Chat labeling templates**](/templates/gallery_chat)
 

docs/source/guide/troubleshooting.md

@@ -38,7 +38,7 @@ To resolve this issue, update the host specified as an environment variable or w
 
 * If you want to upload a large volume of data (thousands of items), consider doing that at a time when people are not labeling or use a different database backend such as PostgreSQL or Redis. You can run Docker Compose from the root directory of Label Studio to use PostgreSQL: `docker-compose up -d`, or see [Sync data from cloud or database storage](storage). 
 
-* If you are using a labeling schema that has many thousands of labels, consider using an [external taxonomy](/tags/taxonomy) instead. 
+* If you are using a labeling schema that has many thousands of labels, consider using an [external taxonomy](/tags/taxonomy.html) instead. 
 
 ### Image/audio/resource loading error while labeling
 

docs/source/templates/chat_eval.md

@@ -97,7 +97,7 @@ The `Chat` tag provides an interface where the annotator can type and send messa
 
   You can customize this parameter with `user`, `assistant`, `system`, `tool`, and `developer`. 
 
-For more information and additional parameters, see the [Chat tag](/tags/chat). 
+For more information and additional parameters, see the [Chat tag](/tags/chat.html). 
 
 
 ## Input data
@@ -146,11 +146,11 @@ You can also import demo chat messages as follows:
 !!! attention
     The chat messages that you import are not selectable. This means that you cannot edit them or apply annotations (ratings, choices, etc) to them.
 
-    You can only select and annotate messages that are added to the chat by an annotator or that are imported as [predictions](/tags/chat#Prediction-format).
+    You can only select and annotate messages that are added to the chat by an annotator or that are imported as [predictions](/tags/chat.html#Prediction-format).
 
 ## Related tags
 
-* [Chat](/tags/chat)
+* [Chat](/tags/chat.html)
 * [Style](/tags/style)
 * [View](/tags/view)
 * [Text](/tags/text)

docs/source/templates/chat_llm_eval.md

@@ -16,7 +16,7 @@ While this template focuses on conversation-level evaluation, you can modify it
 !!! error Enterprise
     This template requires Label Studio Enterprise. 
 
-    Starter Cloud users can use the `Chat` tag, but have limited access to LLM integration. Instead, you can conduct a manual chat or import messages as predictions. See the [Chat tag documentation](/tags/chat#Prediction-format). 
+    Starter Cloud users can use the `Chat` tag, but have limited access to LLM integration. Instead, you can conduct a manual chat or import messages as predictions. See the [Chat tag documentation](/tags/chat.html#Prediction-format). 
 
     For Community users, see our [Conversation AI templates](gallery_conversational_ai) or the [Multi-Turn Chat Evaluation template](multi_turn_chat). 
 
@@ -163,15 +163,15 @@ The `Chat` tag provides an interface where the annotator can type and send messa
 
 * `value`: This is required, and should use a variable referencing your [input data](#Input-data). In this example, we use `$chat` because the input JSON uses `"chat"`.
 
-* `llm`: Messages from the annotator will be sent to an LLM and the response returned within the chat area of the labeling configuration. For more information, see [Chat tag - Use with an LLM](/tags/chat#Use-with-an-LLM). 
+* `llm`: Messages from the annotator will be sent to an LLM and the response returned within the chat area of the labeling configuration. For more information, see [Chat tag - Use with an LLM](/tags/chat.html#Use-with-an-LLM). 
 
 * `minMessages`: The minimum number of messages users must submit to complete the task. You can also set a maximum. 
 
     Both minimum and maximum can also be set in the task data, allowing you to have different limits for each task. For an example, see [Chatbot Evaluation](chatbot#Chat).  
 
 * `editable`: Messages from the annotator and from the LLM are editable. To modify this so that only messages from certain roles are editable, you can specify them (for example, `editable="user,assistant"`). 
 
-For more information and additional parameters, see the [Chat tag](/tags/chat). 
+For more information and additional parameters, see the [Chat tag](/tags/chat.html). 
 
 ### Choices and TextArea
 
@@ -224,11 +224,11 @@ You can also import demo chat messages as follows:
 !!! attention
     The chat messages that you import are not selectable. This means that you cannot edit them or apply annotations (ratings, choices, etc) to them.
 
-    You can only select and annotate messages that are added to the chat by an annotator or that are imported as [predictions](/tags/chat#Prediction-format).
+    You can only select and annotate messages that are added to the chat by an annotator or that are imported as [predictions](/tags/chat.html#Prediction-format).
 
 ## Related tags
 
-* [Chat](/tags/chat)
+* [Chat](/tags/chat.html)
 * [Style](/tags/style)
 * [Text](/tags/text)
 * [View](/tags/view)

docs/source/templates/chat_red_team.md

@@ -14,7 +14,7 @@ Stress‑test your GenAI agent with structured red‑teaming. Use this template
 !!! error Enterprise
     This template requires Label Studio Enterprise. 
 
-    Starter Cloud users can use the `Chat` tag, but have limited access to LLM integration. Instead, you can conduct a manual chat or import messages as predictions. See the [Chat tag documentation](/tags/chat#Prediction-format). 
+    Starter Cloud users can use the `Chat` tag, but have limited access to LLM integration. Instead, you can conduct a manual chat or import messages as predictions. See the [Chat tag documentation](/tags/chat.html#Prediction-format). 
 
     For Community users, see our [Conversation AI templates](gallery_conversational_ai) or the [Multi-Turn Chat Evaluation template](multi_turn_chat).
 
@@ -175,15 +175,15 @@ The `Chat` tag provides an interface where the annotator can type and send messa
 
 * `value`: This is required, and should use a variable referencing your [input data](#Input-data). In this example, we use `$chat` because the input JSON uses `"chat"`.
 
-* `llm`: Messages from the annotator will be sent to an LLM and the response returned within the chat area of the labeling configuration. For more information, see [Chat tag - Use with an LLM](/tags/chat#Use-with-an-LLM). 
+* `llm`: Messages from the annotator will be sent to an LLM and the response returned within the chat area of the labeling configuration. For more information, see [Chat tag - Use with an LLM](/tags/chat.html#Use-with-an-LLM). 
 
 * `minMessages`: The minimum number of messages users must submit to complete the task. You can also set a maximum. 
 
     Both minimum and maximum can also be set in the task data, allowing you to have different limits for each task. For an example, see [Chatbot Evaluation](chatbot#Chat).  
 
 * `editable`: In this example, you are not allowing the annotator to edit messages. You can set this to `true` or modify it so that only messages from certain roles are editable (for example, `editable="user,assistant"`). 
 
-For more information and additional parameters, see the [Chat tag](/tags/chat). 
+For more information and additional parameters, see the [Chat tag](/tags/chat.html). 
 
 ### Per-message evaluation
 
@@ -263,11 +263,11 @@ You can also import demo chat messages as follows:
 !!! attention
     The chat messages that you import are not selectable. This means that you cannot edit them or apply annotations (ratings, choices, etc) to them.
 
-    You can only select and annotate messages that are added to the chat by an annotator or that are imported as [predictions](/tags/chat#Prediction-format).
+    You can only select and annotate messages that are added to the chat by an annotator or that are imported as [predictions](/tags/chat.html#Prediction-format).
 
 ## Related tags
 
-* [Chat](/tags/chat)
+* [Chat](/tags/chat.html)
 * [Style](/tags/style)
 * [View](/tags/view)
 * [Text](/tags/text)

docs/source/templates/chat_rlhf.md

@@ -195,10 +195,10 @@ The `Chat` tag provides an interface where the annotator can type and send messa
 
 * `editable`: In this example, you are not allowing the annotator to edit messages. 
 
-For more information and additional parameters, see the [Chat tag](/tags/chat). 
+For more information and additional parameters, see the [Chat tag](/tags/chat.html). 
 
 !!! note
-    This template is designed to be used to evaluate an imported conversation, so you will likely want to import messages from an external source as [predictions](/tags/chat#Prediction-format). 
+    This template is designed to be used to evaluate an imported conversation, so you will likely want to import messages from an external source as [predictions](/tags/chat.html#Prediction-format). 
 
 ### Conversation evaluation
 
@@ -296,7 +296,7 @@ You can also import demo chat messages as follows:
 
 ### Predictions
 
-If you want to be able to select messages and evaluate them, then you can use [predictions](/tags/chat#Prediction-format). For example:
+If you want to be able to select messages and evaluate them, then you can use [predictions](/tags/chat.html#Prediction-format). For example:
 
 ```json
 [
@@ -346,7 +346,7 @@ If you want to be able to select messages and evaluate them, then you can use [p
 
 ## Related tags
 
-* [Chat](/tags/chat)
+* [Chat](/tags/chat.html)
 * [Style](/tags/style)
 * [View](/tags/view)
 * [Text](/tags/text)

docs/source/templates/chatbot.md

@@ -195,10 +195,10 @@ The `Chat` tag provides an interface where the annotator can type and send messa
 
 * `editable`: In this example, you are not allowing the annotator to edit messages. 
 
-For more information and additional parameters, see the [Chat tag](/tags/chat). 
+For more information and additional parameters, see the [Chat tag](/tags/chat.html). 
 
 !!! note
-    This template is designed to be used to evaluate an imported conversation, so you will likely want to import messages from an external source as [predictions](/tags/chat#Prediction-format). 
+    This template is designed to be used to evaluate an imported conversation, so you will likely want to import messages from an external source as [predictions](/tags/chat.html#Prediction-format). 
 
 ### Per-message evaluation
 
@@ -275,7 +275,7 @@ You can also import demo chat messages as follows:
 
 ### Predictions
 
-If you want to be able to select messages and evaluate them, then you can use [predictions](/tags/chat#Prediction-format). For example:
+If you want to be able to select messages and evaluate them, then you can use [predictions](/tags/chat.html#Prediction-format). For example:
 
 
 ```json
@@ -326,7 +326,7 @@ If you want to be able to select messages and evaluate them, then you can use [p
 
 ## Related tags
 
-* [Chat](/tags/chat)
+* [Chat](/tags/chat.html)
 * [Style](/tags/style)
 * [Text](/tags/text)
 * [View](/tags/view)