Skip to content

Commit c2beec6

Browse files
v-regandownerv-regandowner
authored
Merge pull request #2399 from eric-urban/eur/model-inference-PR1
model inference pr1
2 parents e14999c + 21ba9ad commit c2beec6

File tree

73 files changed

+4836
-88
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

73 files changed

+4836
-88
lines changed

‎.openpublishing.publish.config.json

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -176,6 +176,12 @@
176176
"branch": "main",
177177
"branch_mapping": {}
178178
},
179+
{
180+
"path_to_root": "azureai-model-inference-bicep",
181+
"url": "https://github.com/Azure-Samples/azureai-model-inference-bicep",
182+
"branch": "main",
183+
"branch_mapping": {}
184+
},
179185
{
180186
"path_to_root": "azure-docs-pr-policy-includes",
181187
"url": "https://github.com/MicrosoftDocs/azure-docs-pr",

‎articles/ai-foundry/model-inference/breadcrumb/toc.yml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
- name: Azure
2+
tocHref: /azure/
3+
topicHref: /azure/index
4+
items:
5+
- name: Azure AI services
6+
tocHref: /azure/ai-services/
7+
topicHref: /azure/ai-services/index
8+
items:
9+
- name: Azure AI models in Azure AI Services
10+
tocHref: /azure/ai-services/
11+
topicHref: /azure/ai-services/model-inference/index

‎articles/ai-foundry/model-inference/context/context.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
### YamlMime: ContextObject
2+
brand: azure
3+
breadcrumb_path: ../breadcrumb/toc.yml
4+
toc_rel: ../toc.yml

‎articles/ai-foundry/model-inference/includes/code-create-chat-client-entra.md

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
---
2+
manager: nitinme
3+
ms.service: azure-ai-model-inference
4+
ms.topic: include
5+
ms.date: 1/21/2025
6+
ms.author: fasantia
7+
author: santiagxf
8+
---
9+
10+
# [Python](#tab/python)
11+
12+
Install the package `azure-ai-inference` using your package manager, like pip:
13+
14+
```bash
15+
pip install azure-ai-inference>=1.0.0b5
16+
```
17+
18+
> [!WARNING]
19+
> Azure AI Services resource requires the version `azure-ai-inference>=1.0.0b5` for Python.
20+
21+
Then, you can use the package to consume the model. The following example shows how to create a client to consume chat completions with Entra ID:
22+
23+
```python
24+
import os
25+
from azure.ai.inference import ChatCompletionsClient
26+
from azure.identity import AzureDefaultCredential
27+
28+
model = ChatCompletionsClient(
29+
endpoint=os.environ["AZUREAI_ENDPOINT_URL"],
30+
credential=AzureDefaultCredential(),
31+
)
32+
```
33+
34+
# [JavaScript](#tab/javascript)
35+
36+
Install the package `@azure-rest/ai-inference` using npm:
37+
38+
```bash
39+
npm install @azure-rest/ai-inference
40+
```
41+
42+
Then, you can use the package to consume the model. The following example shows how to create a client to consume chat completions with Entra ID:
43+
44+
```javascript
45+
import ModelClient from "@azure-rest/ai-inference";
46+
import { isUnexpected } from "@azure-rest/ai-inference";
47+
import { AzureDefaultCredential } from "@azure/identity";
48+
49+
const client = new ModelClient(
50+
process.env.AZUREAI_ENDPOINT_URL,
51+
new AzureDefaultCredential()
52+
);
53+
```
54+
55+
# [C#](#tab/csharp)
56+
57+
Install the Azure AI inference library with the following command:
58+
59+
```dotnetcli
60+
dotnet add package Azure.AI.Inference --prerelease
61+
```
62+
63+
Install the `Azure.Identity` package:
64+
65+
```dotnetcli
66+
dotnet add package Azure.Identity
67+
```
68+
69+
Import the following namespaces:
70+
71+
```csharp
72+
using Azure;
73+
using Azure.Identity;
74+
using Azure.AI.Inference;
75+
```
76+
77+
Then, you can use the package to consume the model. The following example shows how to create a client to consume chat completions with Entra ID:
78+
79+
```csharp
80+
ChatCompletionsClient client = new ChatCompletionsClient(
81+
new Uri(Environment.GetEnvironmentVariable("AZURE_INFERENCE_ENDPOINT")),
82+
new DefaultAzureCredential(includeInteractiveCredentials: true)
83+
);
84+
```
85+
86+
# [REST](#tab/rest)
87+
88+
Use the reference section to explore the API design and which parameters are available and indicate authentication token in the header `Authorization`. For example, the reference section for [Chat completions](reference-model-inference-chat-completions.md) details how to use the route `/chat/completions` to generate predictions based on chat-formatted instructions. Notice that the path `/models` is included to the root of the URL:
89+
90+
__Request__
91+
92+
```HTTP/1.1
93+
POST models/chat/completions?api-version=2024-04-01-preview
94+
Authorization: Bearer <bearer-token>
95+
Content-Type: application/json
96+
```
97+
---

‎articles/ai-foundry/model-inference/includes/code-create-chat-client.md

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
---
2+
manager: nitinme
3+
ms.service: azure-ai-model-inference
4+
ms.topic: include
5+
ms.date: 1/21/2025
6+
ms.author: fasantia
7+
author: santiagxf
8+
---
9+
10+
# [Python](#tab/python)
11+
12+
Install the package `azure-ai-inference` using your package manager, like pip:
13+
14+
```bash
15+
pip install azure-ai-inference>=1.0.0b5
16+
```
17+
18+
> [!WARNING]
19+
> Azure AI Services resource requires the version `azure-ai-inference>=1.0.0b5` for Python.
20+
21+
Then, you can use the package to consume the model. The following example shows how to create a client to consume chat completions:
22+
23+
```python
24+
import os
25+
from azure.ai.inference import ChatCompletionsClient
26+
from azure.core.credentials import AzureKeyCredential
27+
28+
model = ChatCompletionsClient(
29+
endpoint=os.environ["AZUREAI_ENDPOINT_URL"],
30+
credential=AzureKeyCredential(os.environ["AZUREAI_ENDPOINT_KEY"]),
31+
)
32+
```
33+
34+
Explore our [samples](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/ai/azure-ai-inference/samples) and read the [API reference documentation](https://aka.ms/azsdk/azure-ai-inference/python/reference) to get yourself started.
35+
36+
# [JavaScript](#tab/javascript)
37+
38+
Install the package `@azure-rest/ai-inference` using npm:
39+
40+
```bash
41+
npm install @azure-rest/ai-inference
42+
```
43+
44+
Then, you can use the package to consume the model. The following example shows how to create a client to consume chat completions:
45+
46+
```javascript
47+
import ModelClient from "@azure-rest/ai-inference";
48+
import { isUnexpected } from "@azure-rest/ai-inference";
49+
import { AzureKeyCredential } from "@azure/core-auth";
50+
51+
const client = new ModelClient(
52+
process.env.AZUREAI_ENDPOINT_URL,
53+
new AzureKeyCredential(process.env.AZUREAI_ENDPOINT_KEY)
54+
);
55+
```
56+
57+
Explore our [samples](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/ai/ai-inference-rest/samples) and read the [API reference documentation](/javascript/api/@azure-rest/ai-inference) to get yourself started.
58+
59+
# [C#](#tab/csharp)
60+
61+
Install the Azure AI inference library with the following command:
62+
63+
```dotnetcli
64+
dotnet add package Azure.AI.Inference --prerelease
65+
```
66+
67+
Import the following namespaces:
68+
69+
```csharp
70+
using Azure;
71+
using Azure.Identity;
72+
using Azure.AI.Inference;
73+
```
74+
75+
Then, you can use the package to consume the model. The following example shows how to create a client to consume chat completions:
76+
77+
```csharp
78+
ChatCompletionsClient client = new ChatCompletionsClient(
79+
new Uri(Environment.GetEnvironmentVariable("AZURE_INFERENCE_ENDPOINT")),
80+
new AzureKeyCredential(Environment.GetEnvironmentVariable("AZURE_INFERENCE_CREDENTIAL"))
81+
);
82+
```
83+
84+
Explore our [samples](https://aka.ms/azsdk/azure-ai-inference/csharp/samples) and read the [API reference documentation](https://aka.ms/azsdk/azure-ai-inference/csharp/reference) to get yourself started.
85+
86+
# [Java](#tab/java)
87+
88+
Add the package to your project:
89+
90+
```xml
91+
<dependency>
92+
<groupId>com.azure</groupId>
93+
<artifactId>azure-ai-inference</artifactId>
94+
<version>1.0.0-beta.1</version>
95+
</dependency>
96+
```
97+
98+
Then, you can use the package to consume the model. The following example shows how to create a client to consume chat completions:
99+
100+
```java
101+
ChatCompletionsClient client = new ChatCompletionsClientBuilder()
102+
.credential(new AzureKeyCredential("{key}"))
103+
.endpoint("{endpoint}")
104+
.buildClient();
105+
```
106+
107+
Explore our [samples](https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/ai/azure-ai-inference/src/samples) and read the [API reference documentation](https://aka.ms/azsdk/azure-ai-inference/java/reference) to get yourself started.
108+
109+
110+
# [REST](#tab/rest)
111+
112+
Use the reference section to explore the API design and which parameters are available. For example, the reference section for [Chat completions](../../../ai-studio/reference/reference-model-inference-chat-completions.md) details how to use the route `/chat/completions` to generate predictions based on chat-formatted instructions. Notice that the path `/models` is included to the root of the URL:
113+
114+
__Request__
115+
116+
```HTTP/1.1
117+
POST models/chat/completions?api-version=2024-04-01-preview
118+
Authorization: Bearer <bearer-token>
119+
Content-Type: application/json
120+
```
121+
---

‎articles/ai-foundry/model-inference/includes/code-create-chat-completion.md

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
---
2+
manager: nitinme
3+
ms.service: azure-ai-model-inference
4+
ms.topic: include
5+
ms.date: 1/21/2025
6+
ms.author: fasantia
7+
author: santiagxf
8+
---
9+
10+
# [Python](#tab/python)
11+
12+
```python
13+
from azure.ai.inference.models import SystemMessage, UserMessage
14+
15+
response = client.complete(
16+
messages=[
17+
SystemMessage(content="You are a helpful assistant."),
18+
UserMessage(content="Explain Riemann's conjecture in 1 paragraph"),
19+
],
20+
model="mistral-large"
21+
)
22+
23+
print(response.choices[0].message.content)
24+
```
25+
26+
# [JavaScript](#tab/javascript)
27+
28+
```javascript
29+
var messages = [
30+
{ role: "system", content: "You are a helpful assistant" },
31+
{ role: "user", content: "Explain Riemann's conjecture in 1 paragraph" },
32+
];
33+
34+
var response = await client.path("/chat/completions").post({
35+
body: {
36+
messages: messages,
37+
model: "mistral-large"
38+
}
39+
});
40+
41+
console.log(response.choices[0].message.content)
42+
```
43+
44+
# [C#](#tab/csharp)
45+
46+
```csharp
47+
requestOptions = new ChatCompletionsOptions()
48+
{
49+
Messages = {
50+
new ChatRequestSystemMessage("You are a helpful assistant."),
51+
new ChatRequestUserMessage("Explain Riemann's conjecture in 1 paragraph")
52+
},
53+
Model = "mistral-large"
54+
};
55+
56+
response = client.Complete(requestOptions);
57+
Console.WriteLine($"Response: {response.Value.Content}");
58+
```
59+
60+
# [Java](#tab/java)
61+
62+
```java
63+
List<ChatRequestMessage> chatMessages = new ArrayList<>();
64+
chatMessages.add(new ChatRequestSystemMessage("You are a helpful assistant"));
65+
chatMessages.add(new ChatRequestUserMessage("Explain Riemann's conjecture in 1 paragraph"));
66+
67+
ChatCompletions chatCompletions = client.complete(new ChatCompletionsOptions(chatMessages));
68+
69+
for (ChatChoice choice : chatCompletions.getChoices()) {
70+
ChatResponseMessage message = choice.getMessage();
71+
System.out.println("Response:" + message.getContent());
72+
}
73+
```
74+
75+
# [REST](#tab/rest)
76+
77+
__Request__
78+
79+
```HTTP/1.1
80+
POST models/chat/completions?api-version=2024-04-01-preview
81+
Authorization: Bearer <bearer-token>
82+
Content-Type: application/json
83+
```
84+
85+
```JSON
86+
{
87+
"messages": [
88+
{
89+
"role": "system",
90+
"content": "You are a helpful assistant"
91+
},
92+
{
93+
"role": "user",
94+
"content": "Explain Riemann's conjecture in 1 paragraph"
95+
}
96+
],
97+
"model": "mistral-large"
98+
}
99+
```
100+
101+
---

0 commit comments

Comments
 (0)