You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Create a notebook with the default python kernel where the python library in the setup section.
110
118
@@ -140,11 +148,11 @@ model = (DataScienceModel()
140
148
).create(model_by_reference=True)
141
149
```
142
150
143
-
# Deploy embedding model
151
+
##Deploy embedding model
144
152
145
153
In order to deploy the model we just created, we set up the infrastructure and container runtime first.
146
154
147
-
## Import Model Deployment Modules
155
+
###Import Model Deployment Modules
148
156
149
157
```
150
158
from ads.model.deployment import (
@@ -155,7 +163,7 @@ from ads.model.deployment import (
155
163
)
156
164
```
157
165
158
-
## Setup Model Deployment Infrastructure
166
+
###Setup Model Deployment Infrastructure
159
167
160
168
```
161
169
infrastructure = (
@@ -177,7 +185,7 @@ infrastructure = (
177
185
)
178
186
```
179
187
180
-
## Configure Model Deployment Runtime
188
+
###Configure Model Deployment Runtime
181
189
182
190
We set the `MODEL_DEPLOY_PREDICT_ENDPOINT` endpoint environment variable with `/v1/embeddings` so that we can
183
191
access the corresponding endpoint from the TEI container. One additional configuration we need to add is `cmd_var`, which
@@ -204,7 +212,7 @@ container_runtime = (
204
212
)
205
213
```
206
214
207
-
## Deploy Model Using Container Runtime
215
+
###Deploy Model Using Container Runtime
208
216
209
217
Once the infrastructure and runtime is configured, we can deploy the model.
210
218
```
@@ -217,7 +225,7 @@ deployment = (
217
225
).deploy(wait_for_completion=False)
218
226
```
219
227
220
-
# Inference
228
+
##Inference
221
229
222
230
Once the model deployment has reached the Active state, we can invoke the model deployment endpoint to interact with the LLM.
223
231
More details on different ways for accessing MD endpoints is documented [here](https://github.com/oracle-samples/oci-data-science-ai-samples/blob/main/ai-quick-actions/model-deployment-tips.md#inferencing-model).
@@ -262,7 +270,7 @@ The raw output (response) has an array of three lists with embedding for the abo
0 commit comments