Skip to content

Commit 4ac0545

Browse files
Model page
Small fixes in Retrain Trigger page
1 parent a31a936 commit 4ac0545

File tree

3 files changed

+59
-44
lines changed

3 files changed

+59
-44
lines changed

md-docs/user_guide/integrations/retrain_triggers.md renamed to md-docs/user_guide/integrations/retrain_trigger.md

Lines changed: 25 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,11 @@
1-
This section offers an overview of setting up retrain triggers for your models. These triggers enable the automatic initiation of your retraining pipeline from the ML cube Platform.
1+
# Retrain Trigger
22

3-
A retrain trigger can be utilized within a Detection Event Rule. When specific criteria are met, it automatically generates the retrain report and activates the trigger. Alternatively, you can manually activate the trigger for the model on the retraining tool page.
3+
This section offers an overview of how you can set up a retrain trigger for your model.
4+
Retrain triggers enable the automatic initiation of your retraining pipeline from the ML cube Platform. They are designed as
5+
integrations with external services and thus require credentials with the appropriate privileges to be executed.
46

5-
A retrain trigger is designed as an integration with an external service and necessitates credentials with the appropriate privileges to execute the action.
7+
A Retrain Trigger can be utilized within a [Detection Event Rule](../monitoring/detection_event_rules.md). Alternatively,
8+
it can be manually activated from the WebApp, in the Retraining section.
69

710
## Supported Triggers
811

@@ -94,11 +97,15 @@ The following retrain triggers are supported:
9497

9598
**Retrain Trigger Setup**
9699
97-
To integrate Amazon Event Bridge, you will need to create a set of AWS credentials, and add a policy that allows to put events in your event bus. Please refer to [this page](index.md) to know more.
100+
To integrate Amazon Event Bridge, you need to create a set of AWS credentials, and add a policy that allows to put events in
101+
your event bus. Please refer to [this page](index.md) for more information.
98102

99-
Once the credentials and the policy have been created, you can set up the retrain trigger for your model through the SDK or the web application.
103+
Once the credentials and the policy have been created, you can set up the retrain trigger for your model through the SDK
104+
or the web application.
100105

101-
!!! example
106+
??? code-block "SDK Example"
107+
108+
Here is an example of how to set up an AWS Event Bridge Retrain Trigger using the SDK:
102109

103110
```py
104111
client.set_retrain_trigger(
@@ -147,11 +154,15 @@ The following retrain triggers are supported:
147154

148155
**Retrain Trigger Setup**
149156

150-
To integrate GCP Pub/Sub, you will need to create a set of GCP credentials, and add a policy that allows to put events in your Pub/Sub topic. Please refer to [this page](index.md) to know more.
157+
To integrate GCP Pub/Sub, you need to create a set of GCP credentials, and add a policy that allows to put events
158+
in your Pub/Sub topic. Please refer to [this page](index.md) for more information.
151159

152-
Once the credentials and the policy have been created, you can set up the retrain trigger for your model through the SDK or the web application.
160+
Once the credentials and the policy have been created, you can set up the retrain trigger for your model through the SDK or
161+
the web application.
153162

154-
!!! example
163+
??? code-block "SDK Example"
164+
165+
Here is an example of how to set up a GCP Pub/Sub Retrain Trigger using the SDK:
155166

156167
```py
157168
client.set_retrain_trigger(
@@ -198,11 +209,14 @@ The following retrain triggers are supported:
198209

199210
**Retrain Trigger Setup**
200211

201-
To integrate Azure Event Grid, you will need to create a set of Azure credentials, and add a role that allows to publish events in your Event Grid topic. Please refer to [this page](index.md) to know more.
212+
To integrate Azure Event Grid, you need to create a set of Azure credentials, and add a role that allows to publish events in your Event Grid topic.
213+
Please refer to [this page](index.md) for more information.
202214

203215
Once the credentials and the policy have been created, you can set up the retrain trigger for your model through the SDK or the web application.
204216

205-
!!! example
217+
??? code-block "SDK Example"
218+
219+
Here is an example of how to set up an Azure Event Grid Retrain Trigger using the SDK:
206220

207221
```py
208222
client.set_retrain_trigger(

md-docs/user_guide/model.md

Lines changed: 33 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,8 @@ In the ML Cube Platform, a Model is a representation of the actual machine learn
44
for its training usually represent the reference data distribution, while production data comprises the data on which the model
55
performs inference.
66

7-
A Model is uniquely associated with a [Task] and it can be created both through the WebApp and the Python SDK. Currently, we only support one model
8-
per Task.
7+
A Model is uniquely associated with a [Task] and it can be created both through the WebApp and the Python SDK.
8+
Currently, we support only one model per Task.
99

1010
A Model is defined by a name and a version. The version is updated whenever the model is retrained, allowing to
1111
track the latest version of the model and the data used for its training. When predictions are uploaded to the platform,
@@ -14,21 +14,21 @@ predictions are associated to the correct model version.
1414

1515
!!! note
1616
You don't need to upload the **real** model on the Platform. We only require its training data and predictions.
17-
The entity you create on the Platform serves more as a placeholder for the model. For this reason,
17+
The entity you create on the Platform serves more as a placeholder for the actual model. For this reason,
1818
the ML cube Platform is considered *model agnostic*.
1919

2020

2121
### RAG Model
2222

2323
RAG Tasks represent an exception to the model framework presented before. In this type of Tasks, the model
2424
is a Large Language Model (LLM), that is used to generate responses to user queries. The model is not trained on a specific dataset
25-
but is rather a pre-trained model that is fine-tuned on the user's data, which means that the classic process of training and
25+
but is rather a pre-trained model, sometimes finetuned on custom domain data, which means that the classic process of training and
2626
retraining does not apply.
2727

2828
To maintain a coherent Model definition across task types, the RAG model is also represented as a Model,
2929
but an update of its version represents an update of the reference data distribution and not necessarily
30-
an update of the model itself. Moreover, most of the attributes which will be described in the following sections
31-
are not applicable, as they are related to the retraining module, which is not usable in RAG tasks.
30+
a retraining of the model itself. Moreover, most of the attributes which will be described in the following sections
31+
are not applicable, as they are related to the retraining module, which is not available for RAG tasks.
3232

3333
### Probabilistic output
3434

@@ -39,45 +39,46 @@ the ML cube Platform will use this information to compute additional metrics and
3939
It is optional and currently supported only for Classification and RAG tasks. If specified, the probabilistic output must be provided
4040
as a new column in the predictions file, following the guidelines in the [Data Schema] page.
4141

42-
### Metric
42+
### Model Metric
4343

4444
A Model Metric represents the evaluation metric used to assess the performance of the model.
45-
It can both represent a performance or an error. The chosen metric will be used in the various views of the platform to
46-
provide insights on the model's performance. The available options are:
45+
It can both represent a performance or an error. The chosen metric will be used in the various views of the WebApp to
46+
provide insights on the model's performance and in the [Performance View](modules/retraining.md#performance-view) section
47+
of the Retraining Module.
4748

48-
- `Accuracy`, for classification tasks
49-
- `RMSE`, for regression tasks
50-
- `R2`, for regression tasks
51-
- `Average Precision`, for Object Detection tasks
49+
The available options are:
50+
51+
| Metric | Task Type |
52+
|-------------------|----------------------------|
53+
| Accuracy | Classification tasks |
54+
| RMSE | Regression tasks |
55+
| R2 | Regression tasks |
56+
| Average Precision | For Object Detection tasks |
5257

5358
RAG tasks have no metric, as in that case the model is an LLM for which classic definitions of metrics are not applicable.
5459

60+
!!! warning
61+
Model Metrics should not be confused with [Monitoring Metrics](monitoring/index.md#monitoring-metrics), which are
62+
entities being monitoring by the ML cube Platform and not necessarily related to a Model.
63+
5564
### Suggestion Type
5665

5766
The Suggestion Type represents the type of suggestion that the ML cube Platform should provide when computing the
58-
[retraining dataset](modules/retraining.md#retraining-dataset). The available options are:
67+
[Retraining Dataset](modules/retraining.md#retraining-dataset). The available options are provided in the related section.
5968

60-
- `Sample Weights`: each sample uploaded in ML cube Platform is assigned a weight that can be used as sample weight in a weighted loss function.
61-
The higher the weight, the greater the importance of the sample for the new retraining.
62-
- `Resampled Dataset`: a list of sample ids (using data schema column object with role ID) is provided indicating which data form the retraining dataset.
63-
This format can be used when the training procedure does not support weighted loss or when a fixed size retraining dataset is preferred.
64-
Note that samples ids can appear more than once: this happens when a sample is particularly important for the new retraining.
6569

66-
[Task]: task.md
67-
[Data Schema]: data_schema.md
70+
### Retraining Cost
6871

69-
[//]: # ()
70-
[//]: # ()
71-
[//]: # (What is additional probabilistic output?)
72+
The Retraining Cost represents the cost associated with retraining the model. This information is used by the Retraining Module
73+
to provide gain-cost analysis and insights on the retraining process. The cost is expressed in the same currency as the one used
74+
in the Task cost information. The default value is 0.0, which means that the cost is negligible.
7275

73-
[//]: # ()
74-
[//]: # (What is metric?)
76+
### Retrain Trigger
7577

76-
[//]: # ()
77-
[//]: # (What is suggestion type?)
78+
You can associate a [Retrain Trigger] to your Model in order to enable the automatic initiation of your retraining pipeline
79+
from the ML cube Platform. More information on how to set up a retrain trigger can be found in the related section.
7880

79-
[//]: # ()
80-
[//]: # (What is retraining cost?)
8181

82-
[//]: # ()
83-
[//]: # (What is retraining trigger?)
82+
[Task]: task.md
83+
[Data Schema]: data_schema.md#subrole
84+
[Retrain Trigger]: integrations/retrain_trigger.md

md-docs/user_guide/monitoring/detection_event_rules.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ data preceding the event, while the second one includes data following the event
4141

4242
Retrain Action enables the automatic retraining of your model. Therefore, it is only available when the target of the rule is related to a model.
4343
The retrain action does not need any parameter because it is automatically inferred from the `model name` attribute of the rule.
44-
Of course, the model must already have a retrain trigger associated before setting up this action.
44+
Of course, the model must already have a [Retrain Trigger](../integrations/retrain_trigger.md) associated before setting up this action.
4545

4646
??? code-block "SDK Example"
4747
The following code demonstrates how to create a rule that matches high severity drift events on the error of a model.

0 commit comments

Comments
 (0)