Skip to content

Commit 30f0aad

Browse files
Add clarifying information & remove mention of mock server, format
1 parent cebe490 commit 30f0aad

File tree

2 files changed

+11
-10
lines changed

2 files changed

+11
-10
lines changed

labs/backend-pool-load-balancing/README.MD

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
[![flow](../../images/backend-pool-load-balancing.gif)](backend-pool-load-balancing.ipynb)
66

7-
Playground to try the built-in load balancing [backend pool functionality of APIM](https://learn.microsoft.com/azure/api-management/backends?tabs=bicep) to either a list of Azure OpenAI endpoints or mock servers.
7+
Playground to try the built-in load balancing [backend pool functionality of APIM](https://learn.microsoft.com/azure/api-management/backends?tabs=bicep) to a list of Azure OpenAI endpoints.
88

99
### Result
1010

labs/backend-pool-load-balancing/backend-pool-load-balancing.ipynb

Lines changed: 10 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -9,12 +9,13 @@
99
"## Backend pool Load Balancing lab\n",
1010
"![flow](../../images/backend-pool-load-balancing.gif)\n",
1111
"\n",
12-
"Playground to try the built-in load balancing [backend pool functionality of APIM](https://learn.microsoft.com/azure/api-management/backends?tabs=bicep) to either a list of Azure OpenAI endpoints.\n",
12+
"Playground to try the built-in load balancing [backend pool functionality of APIM](https://learn.microsoft.com/azure/api-management/backends?tabs=bicep) to a list of Azure OpenAI endpoints.\n",
1313
"\n",
1414
"Notes:\n",
15-
"- The backend pool uses round-robin by default\n",
16-
"- Priority and weight-based routing are also supported: Adjust the `priority` (the lower the number, the higher the priority) and `weight` parameters in the `openai_resources` variable\n",
17-
"- The `retry` API Management policy initiates a retry to an available backend if an HTTP 429 status code is encountered\n",
15+
"- **This is a typical prioritized PTU with fallback consumption scenario**. The lab specifically showcases how a priority 1 (highest) backend is exhausted before gracefully falling back to two equally-weighted priority 2 backends. \n",
16+
"- The backend pool uses round-robin by default.\n",
17+
"- Priority and weight-based routing are supported and can be adjusted by modifying `priority` (the lower the number, the higher the priority) and `weight` parameters in the `openai_resources` variable below.\n",
18+
"- The `retry` API Management policy initiates a retry to an available backend if an HTTP 429 status code is encountered. This is transparent to the caller.\n",
1819
"\n",
1920
"### Result\n",
2021
"![result](result.png)\n",
@@ -132,7 +133,7 @@
132133
"with open(\"policy.xml\", 'r') as policy_xml_file:\n",
133134
" policy_template_xml = policy_xml_file.read()\n",
134135
" if \"{backend-id}\" in policy_template_xml:\n",
135-
" policy_xml = policy_template_xml.replace(\"{backend-id}\", str(\"openai-backend-pool\" if len(openai_resources) > 1 else openai_resources[0].get(\"name\"))) \n",
136+
" policy_xml = policy_template_xml.replace(\"{backend-id}\", str(\"openai-backend-pool\" if len(openai_resources) > 1 else openai_resources[0].get(\"name\")))\n",
136137
" policy_xml_file.close()\n",
137138
"if policy_xml is not None:\n",
138139
" open(\"policy.xml\", 'w').write(policy_xml)\n",
@@ -152,12 +153,12 @@
152153
" }\n",
153154
"}\n",
154155
"\n",
155-
"# write the parameters to a file \n",
156+
"# write the parameters to a file\n",
156157
"with open('params.json', 'w') as bicep_parameters_file:\n",
157158
" bicep_parameters_file.write(json.dumps(bicep_parameters))\n",
158159
"\n",
159160
"# run the deployment\n",
160-
"output = utils.run(f\"az deployment group create --name {deployment_name} --resource-group {resource_group_name} --template-file main.bicep --parameters params.json\", \n",
161+
"output = utils.run(f\"az deployment group create --name {deployment_name} --resource-group {resource_group_name} --template-file main.bicep --parameters params.json\",\n",
161162
" f\"Deployment '{deployment_name}' succeeded\", f\"Deployment '{deployment_name}' failed\")\n",
162163
"open(\"policy.xml\", 'w').write(policy_template_xml)\n",
163164
"\n"
@@ -370,7 +371,7 @@
370371
],
371372
"metadata": {
372373
"kernelspec": {
373-
"display_name": "Python 3",
374+
"display_name": ".venv",
374375
"language": "python",
375376
"name": "python3"
376377
},
@@ -384,7 +385,7 @@
384385
"name": "python",
385386
"nbconvert_exporter": "python",
386387
"pygments_lexer": "ipython3",
387-
"version": "3.12.8"
388+
"version": "3.12.0"
388389
}
389390
},
390391
"nbformat": 4,

0 commit comments

Comments
 (0)