Skip to content

Commit 8e2bcc4

Browse files
authored
Update reference-model-inference-api.md
1 parent 89af9b6 commit 8e2bcc4

File tree

1 file changed

+130
-5
lines changed

1 file changed

+130
-5
lines changed

articles/machine-learning/reference-model-inference-api.md

Lines changed: 130 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -94,7 +94,7 @@ import os
9494
from azure.ai.inference import ChatCompletionsClient
9595
from azure.core.credentials import AzureKeyCredential
9696

97-
model = ChatCompletionsClient(
97+
client = ChatCompletionsClient(
9898
endpoint=os.environ["AZUREAI_ENDPOINT_URL"],
9999
credential=AzureKeyCredential(os.environ["AZUREAI_ENDPOINT_KEY"]),
100100
)
@@ -107,7 +107,7 @@ import os
107107
from azure.ai.inference import ChatCompletionsClient
108108
from azure.identity import AzureDefaultCredential
109109

110-
model = ChatCompletionsClient(
110+
client = ChatCompletionsClient(
111111
endpoint=os.environ["AZUREAI_ENDPOINT_URL"],
112112
credential=AzureDefaultCredential(),
113113
)
@@ -151,6 +151,48 @@ const client = new ModelClient(
151151

152152
Explore our [samples](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/ai/ai-inference-rest/samples) and read the [API reference documentation](https://aka.ms/azsdk/azure-ai-inference/js/reference) to get yourself started.
153153

154+
# [C#](#tab/csharp)
155+
156+
Install the Azure AI inference library with the following command:
157+
158+
```dotnetcli
159+
dotnet add package Azure.AI.Inference --prerelease
160+
```
161+
162+
For endpoint with support for Microsoft Entra ID (formerly Azure Active Directory), install the `Azure.Identity` package:
163+
164+
```dotnetcli
165+
dotnet add package Azure.Identity
166+
```
167+
168+
Import the following namespaces:
169+
170+
```csharp
171+
using Azure;
172+
using Azure.Identity;
173+
using Azure.AI.Inference;
174+
```
175+
176+
Then, you can use the package to consume the model. The following example shows how to create a client to consume chat completions:
177+
178+
```csharp
179+
ChatCompletionsClient client = new ChatCompletionsClient(
180+
new Uri(Environment.GetEnvironmentVariable("AZURE_INFERENCE_ENDPOINT")),
181+
new AzureKeyCredential(Environment.GetEnvironmentVariable("AZURE_INFERENCE_CREDENTIAL"))
182+
);
183+
```
184+
185+
For endpoint with support for Microsoft Entra ID (formerly Azure Active Directory):
186+
187+
```csharp
188+
ChatCompletionsClient client = new ChatCompletionsClient(
189+
new Uri(Environment.GetEnvironmentVariable("AZURE_INFERENCE_ENDPOINT")),
190+
new DefaultAzureCredential(includeInteractiveCredentials: true)
191+
);
192+
```
193+
194+
Explore our [samples](https://aka.ms/azsdk/azure-ai-inference/csharp/samples) and read the [API reference documentation](https://aka.ms/azsdk/azure-ai-inference/csharp/reference) to get yourself started.
195+
154196
# [REST](#tab/rest)
155197

156198
Use the reference section to explore the API design and which parameters are available. For example, the reference section for [Chat completions](reference-model-inference-chat-completions.md) details how to use the route `/chat/completions` to generate predictions based on chat-formatted instructions:
@@ -175,7 +217,7 @@ The following example shows a request passing the parameter `safe_prompt` suppor
175217
# [Python](#tab/python)
176218

177219
```python
178-
response = model.complete(
220+
response = client.complete(
179221
messages=[
180222
SystemMessage(content="You are a helpful assistant."),
181223
UserMessage(content="How many languages are in the world?"),
@@ -210,6 +252,22 @@ var response = await client.path("/chat/completions").post({
210252
console.log(response.choices[0].message.content)
211253
```
212254

255+
# [C#](#tab/csharp)
256+
257+
```csharp
258+
requestOptions = new ChatCompletionsOptions()
259+
{
260+
Messages = {
261+
new ChatRequestSystemMessage("You are a helpful assistant."),
262+
new ChatRequestUserMessage("How many languages are in the world?")
263+
},
264+
AdditionalProperties = { { "logprobs", BinaryData.FromString("true") } },
265+
};
266+
267+
response = client.Complete(requestOptions, extraParams: ExtraParameters.PassThrough);
268+
Console.WriteLine($"Response: {response.Value.Choices[0].Message.Content}");
269+
```
270+
213271
# [REST](#tab/rest)
214272

215273
__Request__
@@ -259,7 +317,7 @@ from azure.ai.inference.models import SystemMessage, UserMessage, ChatCompletion
259317
from azure.core.exceptions import HttpResponseError
260318

261319
try:
262-
response = model.complete(
320+
response = client.complete(
263321
messages=[
264322
SystemMessage(content="You are a helpful assistant."),
265323
UserMessage(content="How many languages are in the world?"),
@@ -314,6 +372,36 @@ catch (error) {
314372
}
315373
```
316374

375+
# [C#](#tab/csharp)
376+
377+
```csharp
378+
try
379+
{
380+
requestOptions = new ChatCompletionsOptions()
381+
{
382+
Messages = {
383+
new ChatRequestSystemMessage("You are a helpful assistant"),
384+
new ChatRequestUserMessage("How many languages are in the world?"),
385+
},
386+
ResponseFormat = new ChatCompletionsResponseFormatJSON()
387+
};
388+
389+
response = client.Complete(requestOptions);
390+
Console.WriteLine(response.Value.Choices[0].Message.Content);
391+
}
392+
catch (RequestFailedException ex)
393+
{
394+
if (ex.Status == 422)
395+
{
396+
Console.WriteLine($"Looks like the model doesn't support a parameter: {ex.Message}");
397+
}
398+
else
399+
{
400+
throw;
401+
}
402+
}
403+
```
404+
317405
# [REST](#tab/rest)
318406

319407
__Request__
@@ -373,7 +461,7 @@ from azure.ai.inference.models import AssistantMessage, UserMessage, SystemMessa
373461
from azure.core.exceptions import HttpResponseError
374462

375463
try:
376-
response = model.complete(
464+
response = client.complete(
377465
messages=[
378466
SystemMessage(content="You are an AI assistant that helps people find information."),
379467
UserMessage(content="Chopping tomatoes and cutting them into cubes or wedges are great ways to practice your knife skills."),
@@ -424,6 +512,37 @@ catch (error) {
424512
}
425513
```
426514

515+
# [C#](#tab/csharp)
516+
517+
```csharp
518+
try
519+
{
520+
requestOptions = new ChatCompletionsOptions()
521+
{
522+
Messages = {
523+
new ChatRequestSystemMessage("You are an AI assistant that helps people find information."),
524+
new ChatRequestUserMessage(
525+
"Chopping tomatoes and cutting them into cubes or wedges are great ways to practice your knife skills."
526+
),
527+
},
528+
};
529+
530+
response = client.Complete(requestOptions);
531+
Console.WriteLine(response.Value.Choices[0].Message.Content);
532+
}
533+
catch (RequestFailedException ex)
534+
{
535+
if (ex.ErrorCode == "content_filter")
536+
{
537+
Console.WriteLine($"Your query has trigger Azure Content Safeaty: {ex.Message}");
538+
}
539+
else
540+
{
541+
throw;
542+
}
543+
}
544+
```
545+
427546
# [REST](#tab/rest)
428547

429548
__Request__
@@ -480,6 +599,12 @@ The client library `@azure-rest/ai-inference` does inference, including chat com
480599

481600
Explore our [samples](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/ai/ai-inference-rest/samples) and read the [API reference documentation](https://aka.ms/azsdk/azure-ai-inference/js/reference) to get yourself started.
482601

602+
# [C#](#tab/csharp)
603+
604+
The client library `Azure.Ai.Inference` does inference, including chat completions, for AI models deployed by Azure AI Studio and Azure Machine Learning Studio. It supports Serverless API endpoints and Managed Compute endpoints (formerly known as Managed Online Endpoints).
605+
606+
Explore our [samples](https://aka.ms/azsdk/azure-ai-inference/csharp/samples) and read the [API reference documentation](https://aka.ms/azsdk/azure-ai-inference/csharp/reference) to get yourself started.
607+
483608
# [REST](#tab/rest)
484609

485610
Explore the reference section of the Azure AI model inference API to see parameters and options to consume models, including chat completions models, deployed by Azure AI Studio and Azure Machine Learning Studio. It supports Serverless API endpoints and Managed Compute endpoints (formerly known as Managed Online Endpoints).

0 commit comments

Comments
 (0)