You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/patterns/industrial-edge/application.md
+37-5Lines changed: 37 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -68,23 +68,55 @@ The pipeline will also have created a PR in gitea, such as the following one:
68
68
69
69
Verify that the change is correct on the datacenter in the `manuela-tst-all` line dashboard and if deemed correct, you can merge the PR in gitea which will roll out the change to the production factory!
70
70
71
-
## Application AI model changes with DevOps
71
+
## AI model changes with MLOps
72
72
73
-
1. On the OpenShift console click the nine-box and select`Red Hat OpenShift AI`. The AI console will open, appearing as follows:
73
+
So far, we have looked at automating the deployment of changes to the application's configuration and code. Let's now explore how we can use OpenShift AI to automate the lifecycle of the application's machine learning model, using similar means in terms of CI/CD and GitOps. For this, we'll switch to the persona of a data scientist or ML engineer working on training and deploying the anomaly detection model.
74
+
75
+
### Logging into the OpenShift AI workbench
76
+
77
+
On the OpenShift console click on the nine-box and choose `Red Hat OpenShift AI`. You'll be taken
78
+
to the AI console which will look like the following:
2. Click the `Data Science Projects` on the left sidebar and choose the `ml-development` project. The project will open, containing a couple of workbenches and a model.:
3. Click the `JupyterLab` workbench to open the notebook where this pattern's data analysis is performed. The `manuela-dev` code will be preloaded in the notebook.
86
+
Clicking on the `JupyterLab` workbench you'll be taken to the notebook where we can explore and analyze the machine data and prototype the code for training the anomaly detection model.
82
87
83
-
4. click the left file browser on `manuela-dev/ml-models/anomaly-detection/1-preprocessing.ipynb`:
The `manuela-dev` code will be preloaded in the notebook and provide access to the Jupyter notebooks and Python modules that implement the model CI/CD steps. In the file browser on the left, navigate to`manuela-dev/ml-models/anomaly-detection/`. You can double click on the Jupyter notebooks (`.ipynb` files) to see the code and the output of the notebooks:
Notebooks are popular among data scientists for interactive data analysis and machine learning experiments. After opening the notebook, walk through the demonstration by
94
+
pressing play and iterating through the commands in the playbook. Jupyter
95
+
playbooks are interactive and you may make changes and also save those changes.
86
96
87
97
After opening the notebook successfully, walk through the demonstration by pressing play and iterating through the commands in the playbooks. Jupyter playbooks are interactive and you might make changes and also save those changes.
88
98
89
99
Running through all the six notebooks will automatically regenerate the anomaly model, prepare the data for the training and push the changes to the internal
90
100
gitea so the inference service can pick up the new model.
101
+
102
+
### Automated model CI/CD
103
+
104
+
Training machine learning models forproduction use cases usually involves ingesting large volumes of data and training for hours or longer. It's a process that should be executedin an automated fashion for repeatability, scalability, observability, and auditability. And we may want to run this process on a pre-defined schedule, say once a week at a certain time. All of this calls for Pipelines!
105
+
106
+
In the file browser on the left, open the `pipelines` folder. This folder contains Python modules corresponding to the Jupyter notebooks in the parent folder. These modules are intended to be run as scripts within the model CI/CD pipeline. The "gluing" of these steps into a proper pipeline is done within the Elyra pipeline definition file `training.pipeline`. Double clicking this file will open the Elyra pipeline editor and visualize the pipeline steps and their order of execution:
To submit the pipeline, select`Run Pipeline` ("play" icon in the top left corner of the Elyra editor). Select "Ok", and view the confirmation message come up after just a few seconds. You can now navigate to the pipeline screen by selecting `Run Details`in the confirmation screen.
We're now back in the OpenShift AI dashboard and can monitor the progress of our model CI/CD pipeline in real-time. You can select individual tasks to look up details such as pipeline logs, which helps to troubleshoot the pipeline code.
Once the pipeline has finished running, step over to Gitea and look up the `industrial-edge` repository (select `rhdp-deploy` branch if you've provisioned the pattern through RHDP). Notice the new commit indicating a model update in the test environment.
0 commit comments