Skip to content

Commit 3508f3b

Browse files
authored
chore(notebooks): [PART 3] Update GB docs generated from notebooks (#6977)
* Fixed links to the Core setup doc page in the notebooks * Get rid of README.md for feedback/metrics-server as another this file (`docs-gb/notebooks/feedback_reward_custom_metrics.md`) is used instead * Update docs pages generated from the notebooks
1 parent 7853655 commit 3508f3b

26 files changed

+1555
-994
lines changed

docs-gb/notebooks/ambassador_canary.md

Lines changed: 76 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33

44
## Setup Seldon Core
55

6-
Use the setup notebook to [Setup Cluster](../notebooks/seldon-core-setup.md#setup-cluster) with [Ambassador Ingress](../notebooks/seldon-core-setup.md#ambassador) and [Install Seldon Core](../notebooks/seldon-core-setup.md#Install-Seldon-Core). Instructions [also online](../notebooks/seldon-core-setup.md).
6+
Use the setup notebook to [Setup Cluster](https://docs.seldon.ai/seldon-core-1/tutorials/notebooks/seldon-core-setup#setup-cluster) with [Ambassador Ingress](https://docs.seldon.ai/seldon-core-1/tutorials/notebooks/seldon-core-setup#install-ingress).
77

88

99
```python
@@ -14,14 +14,6 @@ Use the setup notebook to [Setup Cluster](../notebooks/seldon-core-setup.md#setu
1414

1515

1616

17-
```python
18-
!kubectl config set-context $(kubectl config current-context) --namespace=seldon
19-
```
20-
21-
Context "kind-ansible" modified.
22-
23-
24-
2517
```python
2618
from IPython.core.magic import register_line_cell_magic
2719

@@ -42,7 +34,7 @@ VERSION
4234

4335

4436

45-
'1.15.0-dev'
37+
'1.19.0-dev'
4638

4739

4840

@@ -82,19 +74,21 @@ spec:
8274

8375

8476
```python
85-
!kubectl create -f model.yaml
77+
!kubectl apply -f model.yaml -n seldon
8678
```
8779

8880
seldondeployment.machinelearning.seldon.io/example created
8981

9082

9183

9284
```python
93-
!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=example -o jsonpath='{.items[0].metadata.name}')
85+
!kubectl wait sdep/example \
86+
--for=condition=ready \
87+
--timeout=120s \
88+
-n seldon
9489
```
9590

96-
Waiting for deployment "example-main-0-classifier" rollout to finish: 0 of 1 updated replicas are available...
97-
deployment "example-main-0-classifier" successfully rolled out
91+
seldondeployment.machinelearning.seldon.io/example condition met
9892

9993

10094
### Get predictions
@@ -106,19 +100,32 @@ from seldon_core.seldon_client import SeldonClient
106100
sc = SeldonClient(deployment_name="example", namespace="seldon")
107101
```
108102

109-
2022-08-22 17:10:39.787195: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
110-
2022-08-22 17:10:39.787229: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
103+
2025-12-04 11:09:46.205128: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
104+
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
105+
E0000 00:00:1764846586.222049 3721772 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
106+
E0000 00:00:1764846586.227263 3721772 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
107+
2025-12-04 11:09:46.246078: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
108+
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
111109

112110

113111
#### REST Request
114112

115113

116114
```python
117-
r = sc.predict(gateway="ambassador", transport="rest")
118-
assert r.success == True
119-
print(r)
115+
from tenacity import retry, stop_after_delay, wait_exponential
116+
117+
@retry(stop=stop_after_delay(300), wait=wait_exponential(multiplier=1, min=0.5, max=5))
118+
def make_prediction():
119+
r = sc.predict(gateway="ambassador", transport="rest")
120+
assert r.success == True
121+
return r
122+
123+
make_prediction()
120124
```
121125

126+
127+
128+
122129
Success:True message:
123130
Request:
124131
meta {
@@ -127,17 +134,18 @@ print(r)
127134
tensor {
128135
shape: 1
129136
shape: 1
130-
values: 0.7321696737551778
137+
values: 0.68551705136886776
131138
}
132139
}
133140

134141
Response:
135-
{'data': {'names': ['proba'], 'tensor': {'shape': [1, 1], 'values': [0.10115132850847434]}}, 'meta': {'requestPath': {'classifier': 'seldonio/mock_classifier:1.15.0-dev'}}}
142+
{'data': {'names': ['proba'], 'tensor': {'shape': [1, 1], 'values': [0.09698790957732062]}}, 'meta': {'requestPath': {'classifier': 'seldonio/mock_classifier:1.19.0-dev'}}}
143+
136144

137145

138146
## Launch Canary
139147

140-
We will now extend the existing graph and add a new predictor as a canary using a new model `seldonio/mock_classifier_rest:1.1`. We will add traffic values to split traffic 75/25 to the main and canary.
148+
We will now extend the existing graph and add a new predictor as a canary using a new model `seldonio/mock_classifier_rest`. We will add traffic values to split traffic 75/25 to the main and canary.
141149

142150

143151
```python
@@ -188,29 +196,34 @@ spec:
188196

189197

190198
```python
191-
!kubectl apply -f canary.yaml
199+
!kubectl apply -f canary.yaml -n seldon
192200
```
193201

194-
Warning: resource seldondeployments/example is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
195202
seldondeployment.machinelearning.seldon.io/example configured
196203

197204

198205

199206
```python
200-
!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=example -o jsonpath='{.items[0].metadata.name}')
201-
!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=example -o jsonpath='{.items[1].metadata.name}')
207+
!kubectl wait sdep/example \
208+
--for=condition=ready \
209+
--timeout=120s \
210+
-n seldon
202211
```
203212

204-
Waiting for deployment "example-canary-0-classifier" rollout to finish: 0 of 1 updated replicas are available...
205-
deployment "example-canary-0-classifier" successfully rolled out
206-
deployment "example-main-0-classifier" successfully rolled out
213+
seldondeployment.machinelearning.seldon.io/example condition met
207214

208215

209216
Show our REST requests are now split with roughly 25% going to the canary.
210217

211218

212219
```python
213-
sc.predict(gateway="ambassador", transport="rest")
220+
@retry(stop=stop_after_delay(300), wait=wait_exponential(multiplier=1, min=0.5, max=5))
221+
def make_prediction():
222+
r = sc.predict(gateway="ambassador", transport="rest")
223+
assert r.success == True
224+
return r
225+
226+
make_prediction()
214227
```
215228

216229

@@ -224,21 +237,23 @@ sc.predict(gateway="ambassador", transport="rest")
224237
tensor {
225238
shape: 1
226239
shape: 1
227-
values: 0.5515112951204143
240+
values: 0.83946571577703166
228241
}
229242
}
230243

231244
Response:
232-
{'data': {'names': ['proba'], 'tensor': {'shape': [1, 1], 'values': [0.08586865749102578]}}, 'meta': {'requestPath': {'classifier': 'seldonio/mock_classifier:1.15.0-dev'}}}
245+
{'data': {'names': ['proba'], 'tensor': {'shape': [1, 1], 'values': [0.11133259692564312]}}, 'meta': {'requestPath': {'classifier': 'seldonio/mock_classifier:1.19.0-dev'}}}
233246

234247

235248

236249

237250
```python
238251
from collections import defaultdict
252+
import time
239253

240254
counts = defaultdict(int)
241255
n = 100
256+
time.sleep(10) # wait before sending requests
242257
for i in range(n):
243258
r = sc.predict(gateway="ambassador", transport="rest")
244259
```
@@ -247,33 +262,51 @@ Following checks number of prediction requests processed by default/canary predi
247262

248263

249264
```python
250-
default_count = !kubectl logs $(kubectl get pod -lseldon-app=example-main -o jsonpath='{.items[0].metadata.name}') classifier | grep "root:predict" | wc -l
251-
```
265+
import time
252266

267+
@retry(stop=stop_after_delay(300), wait=wait_exponential(multiplier=1, min=0.5, max=5))
268+
def get_requests_cound_for_main():
269+
default_count = !kubectl logs -l seldon-app=example-main -n seldon -c classifier --tail 1000 | grep "root:predict" | wc -l
270+
return float(default_count[0])
253271

254-
```python
255-
canary_count = !kubectl logs $(kubectl get pod -lseldon-app=example-canary -o jsonpath='{.items[0].metadata.name}') classifier | grep "root:predict" | wc -l
272+
273+
time.sleep(10) # wait for logs to be flushed
274+
default_count = get_requests_cound_for_main()
275+
print(f"main logs count {default_count}")
256276
```
257277

278+
main logs count 75.0
279+
280+
258281

259282
```python
260-
canary_percentage = float(canary_count[0]) / float(default_count[0])
261-
print(canary_percentage)
262-
assert canary_percentage > 0.1 and canary_percentage < 0.5
283+
@retry(stop=stop_after_delay(300), wait=wait_exponential(multiplier=1, min=0.5, max=5))
284+
def get_requests_cound_for_canary():
285+
canary_count = !kubectl logs -l seldon-app=example-canary -n seldon -c classifier --tail 1000 | grep "root:predict" | wc -l
286+
return float(canary_count[0])
287+
288+
time.sleep(10) # wait for logs to be flushed
289+
canary_count = get_requests_cound_for_canary()
290+
print(f"canary logs count {canary_count}")
263291
```
264292

265-
0.3246753246753247
293+
canary logs count 27.0
266294

267295

268296

269297
```python
270-
!kubectl delete -f canary.yaml
298+
canary_percentage = canary_count / default_count
299+
print(f"canary percentage {canary_percentage}")
300+
assert canary_percentage > 0.1 and canary_percentage < 0.5
271301
```
272302

273-
seldondeployment.machinelearning.seldon.io "example" deleted
303+
canary percentage 0.36
274304

275305

276306

277307
```python
278-
308+
!kubectl delete -f canary.yaml -n seldon
279309
```
310+
311+
seldondeployment.machinelearning.seldon.io "example" deleted
312+

0 commit comments

Comments
 (0)