Skip to content

Add Azure OpenAI Chat and Embedding Options #286

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed

Add Azure OpenAI Chat and Embedding Options #286

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
24e2bb6
Add Azure OpenAI Chat and Embedding Options
tzolov Jan 30, 2024
f38e00e
Rebase
tzolov Jan 30, 2024
63ac7fd
Add protable ChatOptions support
tzolov Feb 2, 2024
2470f54
reabase and doc refs fixes
tzolov Feb 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 4 additions & 73 deletions models/spring-ai-azure-openai/README_AZURE_OPENAI.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,78 +1,9 @@
# 1. Azure OpenAI
# Azure OpenAI
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not remove this readme... docs should be centralized.


Provides Azure OpenAI Chat and Embedding clients.
Leverages the native [OpenAIClient](https://learn.microsoft.com/en-us/java/api/overview/azure/ai-openai-readme?view=azure-java-preview#streaming-chat-completions) to interact with the [Amazon AI Studio models and deployment](https://oai.azure.com/).

## 1.1 Prerequisites
Find additional information:

1. Azure Subscription: You will need an [Azure subscription](https://azure.microsoft.com/en-us/free/) to use any Azure service.
2. Azure AI, Azure OpenAI Service: Create [Azure OpenAI](https://portal.azure.com/#create/Microsoft.CognitiveServicesOpenAI).
Once the service is created, obtain the endpoint and apiKey from the `Keys and Endpoint` section under `Resource Management`.
3. Use the [Azure Ai Studio](https://oai.azure.com/portal) to deploy the models you are going to use.

## 1.2 AzureOpenAiChatClient

[AzureOpenAiChatClient](./src/main/java/org/springframework/ai/azure/openai/AzureOpenAiChatClient.java) implements the Spring-Ai `ChatClient` and `StreamingChatClient` on top of the `OpenAIClient`.

[AzureOpenAiEmbeddingClient](./src/main/java/org/springframework/ai/azure/openai/AzureOpenAiEmbeddingClient.java) implements the Spring-Ai `EmbeddingClient` on top of the `OpenAIClient`.


You can configure the AzureOpenAiChatClient and AzureOpenAiEmbeddingClientlike this:

```java
@Bean
public OpenAIClient openAIClient() {
return new OpenAIClientBuilder()
.credential(new AzureKeyCredential({YOUR_AZURE_OPENAI_API_KEY}))
.endpoint({YOUR_AZURE_OPENAI_ENDPOINT})
.buildClient();
}

@Bean
public AzureOpenAiChatClient cohereChatClient(OpenAIClient openAIClient) {
return new AzureOpenAiChatClient(openAIClient)
.withModel("gpt-35-turbo")
.withMaxTokens(200)
.withTemperature(0.8);
}

@Bean
public AzureOpenAiEmbeddingClient cohereEmbeddingClient(OpenAIClient openAIClient) {
return new AzureOpenAiEmbeddingClient(openAIClient, "text-embedding-ada-002-v1");
}
```

## 1.3 Azure OpenAi Auto-Configuration and Spring Boot Starter

You can leverage the `spring-ai-azure-openai-spring-boot-starter` Boot starter.
For this add the following dependency:

```xml
<dependency>
<artifactId>spring-ai-azure-openai-spring-boot-starter</artifactId>
<groupId>org.springframework.ai</groupId>
<version>0.8.0-SNAPSHOT</version>
</dependency>
```

Use the `AzureOpenAiConnectionProperties` to configure the Azure OpenAI access:

| Property | Description | Default |
| ------------- | ------------- | ------------- |
| spring.ai.azure.openai.apiKey | Azure AI Open AI credentials api key. | From the Azure AI OpenAI `Keys and Endpoint` section under `Resource Management` |
| spring.ai.azure.openai.endpoint | Azure AI Open AI endpoint. | From the Azure AI OpenAI `Keys and Endpoint` section under `Resource Management` |

Use the `AzureOpenAiChatProperties` to configure the Chat client:

| Property | Description | Default |
| ------------- | ------------- | ------------- |
| spring.ai.azure.openai.chat.model | The model id to use. | gpt-35-turbo |
| spring.ai.azure.openai.chat.temperature | Controls the randomness of the output. Values can range over [0.0,1.0] | 0.7 |
| spring.ai.azure.openai.chat.topP | An alternative to sampling with temperature called nucleus sampling. | |
| spring.ai.azure.openai.chat.maxTokens | The maximum number of tokens to generate. | |

Use the `AzureOpenAiEmbeddingProperties` to configure the Embedding client:

| Property | Description | Default |
| ------------- | ------------- | ------------- |
| spring.ai.azure.openai.embedding.model | The model id to use for embedding | text-embedding-ada-002 |
- [Azure OpenAi Chat Client](https://docs.spring.io/spring-ai/reference/api/clients/azure-openai-chat.html)
- [Azure OpenAi Embeddings Client](https://docs.spring.io/spring-ai/reference/api/embeddings/azure-openai-embeddings.html)
Loading