Skip to content

Commit 33a8c2b

Browse files
HJ HangHJ Hang
authored andcommitted
update PS
1 parent ebfed56 commit 33a8c2b

File tree

1 file changed

+3
-7
lines changed

1 file changed

+3
-7
lines changed

AKS-Arc/deploy-ai-model.md

Lines changed: 3 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -167,12 +167,7 @@ kubectl run -it --rm --restart=Never curl --image=curlimages/curl -- curl -X POS
167167

168168
```powershell
169169
$CLUSTERIP = $(kubectl get svc workspace-llm -o jsonpath="{.spec.clusterIPs[0]}" )
170-
$jsonContent = '{
171-
"model": "phi-3.5-mini-instruct",
172-
"prompt": "What is kubernetes?",
173-
"max_tokens": 20,
174-
"temperature": 0
175-
}'
170+
$jsonContent = '{"model":"phi-3.5-mini-instruct","prompt":"What is kubernetes","max_tokens":200,"temperature":0}' | ConvertTo-Json
176171
177172
kubectl run -it --rm --restart=Never curl --image=curlimages/curl -- curl -X POST http://$CLUSTERIP/v1/completions -H "accept: application/json" -H "Content-Type: application/json" -d $jsonContent
178173
```
@@ -208,5 +203,6 @@ The following table shows the supported GPU models and their corresponding VM SK
208203
1. If you see an error during prompt testing such as **{"detail":[{"type":"json_invalid","loc":["body",1],"msg":"JSON decode error","input":{},"ctx":{"error":"Expecting property name enclosed in double quotes"}}]}**, it's possible that your PowerShell terminal version is 5.1. Make sure the terminal version is at least 7.4.
209204

210205
## Next steps
206+
* [Monitor the inference metrics](https://learn.microsoft.com/en-us/azure/aks/ai-toolchain-operator-monitoring) in Managed Prometheus and Managed Grafana
207+
* Learn more about KAITO, visit [KAITO GitHub Repo](https://github.com/kaito-project/kaito)
211208

212-
In this article, you learned how to deploy an AI model on AKS enabled by Azure Arc with the Kubernetes AI toolchain operator (KAITO). For more information about the KAITO project, see the [KAITO GitHub repo](https://github.com/kaito-project/kaito).

0 commit comments

Comments
 (0)