Skip to content

Commit 086d987

Browse files
authored
Update reference-model-inference-api.md
1 parent 34a4532 commit 086d987

File tree

1 file changed

+126
-1
lines changed

1 file changed

+126
-1
lines changed

articles/ai-studio/reference/reference-model-inference-api.md

Lines changed: 126 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ Models deployed to [managed inference](../concepts/deployments-overview.md):
5656
> [!div class="checklist"]
5757
> * [Meta Llama 3 instruct](../how-to/deploy-models-llama.md) family of models
5858
> * [Phi-3](../how-to/deploy-models-phi-3.md) family of models
59-
> * Mixtral famility of models
59+
> * [Mistral](../how-to/deploy-models-mistral-open.md) and [Mixtral](../how-to/deploy-models-mistral-open.md?tabs=mistral-8x7B-instruct) family of models.
6060
6161
The API is compatible with Azure OpenAI model deployments.
6262

@@ -154,6 +154,48 @@ const client = new ModelClient(
154154

155155
Explore our [samples](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/ai/ai-inference-rest/samples) and read the [API reference documentation](https://aka.ms/AAp1kxa) to get yourself started.
156156

157+
# [C#](#tab/csharp)
158+
159+
Install the Azure AI inference library with the following command:
160+
161+
```dotnetcli
162+
dotnet add package Azure.AI.Inference --prerelease
163+
```
164+
165+
For endpoint with support for Microsoft Entra ID (formerly Azure Active Directory), install the `Azure.Identity` package:
166+
167+
```dotnetcli
168+
dotnet add package Azure.Identity
169+
```
170+
171+
Import the following namespaces:
172+
173+
```csharp
174+
using Azure;
175+
using Azure.Identity;
176+
using Azure.AI.Inference;
177+
```
178+
179+
Then, you can use the package to consume the model. The following example shows how to create a client to consume chat completions:
180+
181+
```csharp
182+
ChatCompletionsClient client = new ChatCompletionsClient(
183+
new Uri(Environment.GetEnvironmentVariable("AZURE_INFERENCE_ENDPOINT")),
184+
new AzureKeyCredential(Environment.GetEnvironmentVariable("AZURE_INFERENCE_CREDENTIAL"))
185+
);
186+
```
187+
188+
For endpoint with support for Microsoft Entra ID (formerly Azure Active Directory):
189+
190+
```csharp
191+
ChatCompletionsClient client = new ChatCompletionsClient(
192+
new Uri(Environment.GetEnvironmentVariable("AZURE_INFERENCE_ENDPOINT")),
193+
new DefaultAzureCredential(includeInteractiveCredentials: true)
194+
);
195+
```
196+
197+
Explore our [samples](https://aka.ms/azsdk/azure-ai-inference/csharp/samples) and read the [API reference documentation](https://aka.ms/azsdk/azure-ai-inference/csharp/reference) to get yourself started.
198+
157199
# [REST](#tab/rest)
158200

159201
Use the reference section to explore the API design and which parameters are available. For example, the reference section for [Chat completions](reference-model-inference-chat-completions.md) details how to use the route `/chat/completions` to generate predictions based on chat-formatted instructions:
@@ -215,6 +257,22 @@ var response = await client.path("/chat/completions").post({
215257
console.log(response.choices[0].message.content)
216258
```
217259

260+
# [C#](#tab/csharp)
261+
262+
```csharp
263+
requestOptions = new ChatCompletionsOptions()
264+
{
265+
Messages = {
266+
new ChatRequestSystemMessage("You are a helpful assistant."),
267+
new ChatRequestUserMessage("How many languages are in the world?")
268+
},
269+
AdditionalProperties = { { "logprobs", BinaryData.FromString("true") } },
270+
};
271+
272+
response = client.Complete(requestOptions, extraParams: ExtraParameters.PassThrough);
273+
Console.WriteLine($"Response: {response.Value.Choices[0].Message.Content}");
274+
```
275+
218276
# [REST](#tab/rest)
219277

220278
__Request__
@@ -319,6 +377,36 @@ catch (error) {
319377
}
320378
```
321379

380+
# [C#](#tab/csharp)
381+
382+
```csharp
383+
try
384+
{
385+
requestOptions = new ChatCompletionsOptions()
386+
{
387+
Messages = {
388+
new ChatRequestSystemMessage("You are a helpful assistant"),
389+
new ChatRequestUserMessage("How many languages are in the world?"),
390+
},
391+
ResponseFormat = new ChatCompletionsResponseFormatJSON()
392+
};
393+
394+
response = client.Complete(requestOptions);
395+
Console.WriteLine(response.Value.Choices[0].Message.Content);
396+
}
397+
catch (RequestFailedException ex)
398+
{
399+
if (ex.Status == 422)
400+
{
401+
Console.WriteLine($"Looks like the model doesn't support a parameter: {ex.Message}");
402+
}
403+
else
404+
{
405+
throw;
406+
}
407+
}
408+
```
409+
322410
# [REST](#tab/rest)
323411

324412
__Request__
@@ -429,6 +517,37 @@ catch (error) {
429517
}
430518
```
431519

520+
# [C#](#tab/csharp)
521+
522+
```csharp
523+
try
524+
{
525+
requestOptions = new ChatCompletionsOptions()
526+
{
527+
Messages = {
528+
new ChatRequestSystemMessage("You are an AI assistant that helps people find information."),
529+
new ChatRequestUserMessage(
530+
"Chopping tomatoes and cutting them into cubes or wedges are great ways to practice your knife skills."
531+
),
532+
},
533+
};
534+
535+
response = client.Complete(requestOptions);
536+
Console.WriteLine(response.Value.Choices[0].Message.Content);
537+
}
538+
catch (RequestFailedException ex)
539+
{
540+
if (ex.ErrorCode == "content_filter")
541+
{
542+
Console.WriteLine($"Your query has trigger Azure Content Safeaty: {ex.Message}");
543+
}
544+
else
545+
{
546+
throw;
547+
}
548+
}
549+
```
550+
432551
# [REST](#tab/rest)
433552

434553
__Request__
@@ -485,6 +604,12 @@ The client library `@azure-rest/ai-inference` does inference, including chat com
485604

486605
Explore our [samples](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/ai/ai-inference-rest/samples) and read the [API reference documentation](https://aka.ms/AAp1kxa) to get yourself started.
487606

607+
# [C#](#tab/csharp)
608+
609+
The client library `Azure.Ai.Inference` does inference, including chat completions, for AI models deployed by Azure AI Studio and Azure Machine Learning Studio. It supports Serverless API endpoints and Managed Compute endpoints (formerly known as Managed Online Endpoints).
610+
611+
Explore our [samples](https://aka.ms/azsdk/azure-ai-inference/csharp/samples) and read the [API reference documentation](https://aka.ms/azsdk/azure-ai-inference/csharp/reference) to get yourself started.
612+
488613
# [REST](#tab/rest)
489614

490615
Explore the reference section of the Azure AI model inference API to see parameters and options to consume models, including chat completions models, deployed by Azure AI Studio and Azure Machine Learning Studio. It supports Serverless API endpoints and Managed Compute endpoints (formerly known as Managed Online Endpoints).

0 commit comments

Comments
 (0)