|
| 1 | +# Azure OpenAI client library for Java |
| 2 | + |
| 3 | +Azure OpenAI is a managed service that allows developers to deploy, tune, and generate content from OpenAI models on |
| 4 | +Azure resources. |
| 5 | + |
| 6 | +The Azure OpenAI client library for Java is an adaptation of OpenAI's REST APIs that provides an idiomatic interface |
| 7 | +and rich integration with the rest of the Azure SDK ecosystem. |
| 8 | + |
| 9 | +Use the client library for Azure OpenAI to: |
| 10 | + |
| 11 | +* [Create a completion for text][microsoft_docs_openai_completion] |
| 12 | +* [Create a text embedding for comparisons][microsoft_docs_openai_embedding] |
| 13 | + |
| 14 | +[Source code][source_code] | [API reference documentation][docs] | [Product Documentation][product_documentation] | [Samples][samples_readme] |
| 15 | + |
| 16 | +## Getting started |
| 17 | + |
| 18 | +### Prerequisites |
| 19 | + |
| 20 | +- [Java Development Kit (JDK)][jdk] with version 8 or above |
| 21 | +- [Azure Subscription][azure_subscription] |
| 22 | +- [Azure OpenAI access][azure_openai_access] |
| 23 | +- [Quickstart: Get started generating text using Azure OpenAI Service][quickstart] |
| 24 | + |
| 25 | +### Adding the package to your product |
| 26 | + |
| 27 | +[//]: # ({x-version-update-start;com.azure:azure-ai-openai;current}) |
| 28 | +```xml |
| 29 | +<dependency> |
| 30 | + <groupId>com.azure</groupId> |
| 31 | + <artifactId>azure-ai-openai</artifactId> |
| 32 | + <version>1.0.0-beta.1</version> |
| 33 | +</dependency> |
| 34 | +``` |
| 35 | +[//]: # ({x-version-update-end}) |
| 36 | + |
| 37 | +### Authentication |
| 38 | + |
| 39 | +In order to interact with the Azure OpenAI service you'll need to create an instance of client class, |
| 40 | +[OpenAIAsyncClient][openai_client_async] or [OpenAIClient][openai_client_sync] by using |
| 41 | +[OpenAIClientBuilder][openai_client_builder]. To configure a client for use with |
| 42 | +Azure OpenAI, provide a valid endpoint URI to an Azure OpenAI resource along with a corresponding key credential, |
| 43 | +token credential, or [Azure Identity][azure_identity] credential that's authorized to use the Azure OpenAI resource. |
| 44 | + |
| 45 | +#### Create a Azure OpenAI client with key credential |
| 46 | +Get Azure OpenAI `key` credential from the Azure Portal. |
| 47 | + |
| 48 | +```java readme-sample-createSyncClientKeyCredential |
| 49 | +OpenAIClient client = new OpenAIClientBuilder() |
| 50 | + .credential(new AzureKeyCredential("{key}")) |
| 51 | + .endpoint("{endpoint}") |
| 52 | + .buildClient(); |
| 53 | +``` |
| 54 | +or |
| 55 | +```java readme-sample-createAsyncClientKeyCredential |
| 56 | +OpenAIAsyncClient client = new OpenAIClientBuilder() |
| 57 | + .credential(new AzureKeyCredential("{key}")) |
| 58 | + .endpoint("{endpoint}") |
| 59 | + .buildAsyncClient(); |
| 60 | +``` |
| 61 | + |
| 62 | +#### Create an Azure OpenAI client with Azure Active Directory credential |
| 63 | +Azure SDK for Java supports an Azure Identity package, making it easy to get credentials from Microsoft identity |
| 64 | +platform. |
| 65 | + |
| 66 | +Authentication with AAD requires some initial setup: |
| 67 | +* Add the Azure Identity package |
| 68 | + |
| 69 | +[//]: # ({x-version-update-start;com.azure:azure-identity;dependency}) |
| 70 | +```xml |
| 71 | +<dependency> |
| 72 | + <groupId>com.azure</groupId> |
| 73 | + <artifactId>azure-identity</artifactId> |
| 74 | + <version>1.9.0</version> |
| 75 | +</dependency> |
| 76 | +``` |
| 77 | +[//]: # ({x-version-update-end}) |
| 78 | + |
| 79 | +After setup, you can choose which type of [credential][azure_identity_credential_type] from azure.identity to use. |
| 80 | +As an example, [DefaultAzureCredential][wiki_identity] can be used to authenticate the client: |
| 81 | +Set the values of the client ID, tenant ID, and client secret of the AAD application as environment variables: |
| 82 | +`AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_CLIENT_SECRET`. |
| 83 | + |
| 84 | +Authorization is easiest using [DefaultAzureCredential][wiki_identity]. It finds the best credential to use in its |
| 85 | +running environment. For more information about using Azure Active Directory authorization with OpenAI service, please |
| 86 | +refer to [the associated documentation][aad_authorization]. |
| 87 | + |
| 88 | +```java readme-sample-createOpenAIClientWithAAD |
| 89 | +TokenCredential defaultCredential = new DefaultAzureCredentialBuilder().build(); |
| 90 | +OpenAIClient client = new OpenAIClientBuilder() |
| 91 | + .credential(defaultCredential) |
| 92 | + .endpoint("{endpoint}") |
| 93 | + .buildClient(); |
| 94 | +``` |
| 95 | + |
| 96 | +#### Create a client with proxy options |
| 97 | +Create an OpenAI client with proxy options. |
| 98 | +```java readme-sample-createOpenAIClientWithProxyOption |
| 99 | +// Proxy options |
| 100 | +final String hostname = "{your-host-name}"; |
| 101 | +final int port = 447; // your port number |
| 102 | + |
| 103 | +ProxyOptions proxyOptions = new ProxyOptions(ProxyOptions.Type.HTTP, new InetSocketAddress(hostname, port)) |
| 104 | + .setCredentials("{username}", "{password}"); |
| 105 | + |
| 106 | +OpenAIClient client = new OpenAIClientBuilder() |
| 107 | + .credential(new AzureKeyCredential("{key}")) |
| 108 | + .endpoint("{endpoint}") |
| 109 | + .clientOptions(new HttpClientOptions().setProxyOptions(proxyOptions)) |
| 110 | + .buildClient(); |
| 111 | +``` |
| 112 | + |
| 113 | +## Key concepts |
| 114 | + |
| 115 | +## Examples |
| 116 | +The following sections provide several code snippets covering some of the most common OpenAI service tasks, including: |
| 117 | + |
| 118 | +* [Text completions sample](#text-completions "Text completions") |
| 119 | +* [Streaming text completions sample](#streaming-text-completions "Streaming text completions") |
| 120 | +* [Chat completions sample](#chat-completions "Chat completions") |
| 121 | +* [Streaming chat completions sample](#streaming-chat-completions "Streaming chat completions") |
| 122 | +* [Embeddings sample](#text-embeddings "Text Embeddings") |
| 123 | + |
| 124 | +### Text completions |
| 125 | + |
| 126 | +``` java readme-sample-getCompletions |
| 127 | +List<String> prompt = new ArrayList<>(); |
| 128 | +prompt.add("Say this is a test"); |
| 129 | + |
| 130 | +Completions completions = client.getCompletions("{deploymentOrModelId}", new CompletionsOptions(prompt)); |
| 131 | + |
| 132 | +System.out.printf("Model ID=%s is created at %d.%n", completions.getId(), completions.getCreated()); |
| 133 | +for (Choice choice : completions.getChoices()) { |
| 134 | + System.out.printf("Index: %d, Text: %s.%n", choice.getIndex(), choice.getText()); |
| 135 | +} |
| 136 | +``` |
| 137 | + |
| 138 | +### Streaming text completions |
| 139 | + |
| 140 | +```java readme-sample-getCompletionsStream |
| 141 | +List<String> prompt = new ArrayList<>(); |
| 142 | +prompt.add("How to bake a cake?"); |
| 143 | + |
| 144 | +IterableStream<Completions> completionsStream = client |
| 145 | + .getCompletionsStream("{deploymentOrModelId}", new CompletionsOptions(prompt)); |
| 146 | + |
| 147 | +completionsStream.forEach(completions -> { |
| 148 | + System.out.printf("Model ID=%s is created at %d.%n", completions.getId(), completions.getCreated()); |
| 149 | + for (Choice choice : completions.getChoices()) { |
| 150 | + System.out.printf("Index: %d, Text: %s.%n", choice.getIndex(), choice.getText()); |
| 151 | + } |
| 152 | +}); |
| 153 | +``` |
| 154 | + |
| 155 | +### Chat completions |
| 156 | + |
| 157 | +``` java readme-sample-getChatCompletions |
| 158 | +List<ChatMessage> chatMessages = new ArrayList<>(); |
| 159 | +chatMessages.add(new ChatMessage(ChatRole.SYSTEM).setContent("You are a helpful assistant. You will talk like a pirate.")); |
| 160 | +chatMessages.add(new ChatMessage(ChatRole.USER).setContent("Can you help me?")); |
| 161 | +chatMessages.add(new ChatMessage(ChatRole.ASSISTANT).setContent("Of course, me hearty! What can I do for ye?")); |
| 162 | +chatMessages.add(new ChatMessage(ChatRole.USER).setContent("What's the best way to train a parrot?")); |
| 163 | + |
| 164 | +ChatCompletions chatCompletions = client.getChatCompletions("{deploymentOrModelId}", |
| 165 | + new ChatCompletionsOptions(chatMessages)); |
| 166 | + |
| 167 | +System.out.printf("Model ID=%s is created at %d.%n", chatCompletions.getId(), chatCompletions.getCreated()); |
| 168 | +for (ChatChoice choice : chatCompletions.getChoices()) { |
| 169 | + ChatMessage message = choice.getMessage(); |
| 170 | + System.out.printf("Index: %d, Chat Role: %s.%n", choice.getIndex(), message.getRole()); |
| 171 | + System.out.println("Message:"); |
| 172 | + System.out.println(message.getContent()); |
| 173 | +} |
| 174 | +``` |
| 175 | +Please refer to the service documentation for a conceptual discussion of [text completion][microsoft_docs_openai_completion]. |
| 176 | + |
| 177 | +### Streaming chat completions |
| 178 | + |
| 179 | +```java readme-sample-getChatCompletionsStream |
| 180 | +List<ChatMessage> chatMessages = new ArrayList<>(); |
| 181 | +chatMessages.add(new ChatMessage(ChatRole.SYSTEM).setContent("You are a helpful assistant. You will talk like a pirate.")); |
| 182 | +chatMessages.add(new ChatMessage(ChatRole.USER).setContent("Can you help me?")); |
| 183 | +chatMessages.add(new ChatMessage(ChatRole.ASSISTANT).setContent("Of course, me hearty! What can I do for ye?")); |
| 184 | +chatMessages.add(new ChatMessage(ChatRole.USER).setContent("What's the best way to train a parrot?")); |
| 185 | + |
| 186 | +IterableStream<ChatCompletions> chatCompletionsStream = client.getChatCompletionsStream("{deploymentOrModelId}", |
| 187 | + new ChatCompletionsOptions(chatMessages)); |
| 188 | + |
| 189 | +chatCompletionsStream.forEach(chatCompletions -> { |
| 190 | + System.out.printf("Model ID=%s is created at %d.%n", chatCompletions.getId(), chatCompletions.getCreated()); |
| 191 | + for (ChatChoice choice : chatCompletions.getChoices()) { |
| 192 | + ChatMessageDelta message = choice.getDelta(); |
| 193 | + if (message != null) { |
| 194 | + System.out.printf("Index: %d, Chat Role: %s.%n", choice.getIndex(), message.getRole()); |
| 195 | + System.out.println("Message:"); |
| 196 | + System.out.println(message.getContent()); |
| 197 | + } |
| 198 | + } |
| 199 | + |
| 200 | + CompletionsUsage usage = chatCompletions.getUsage(); |
| 201 | + if (usage != null) { |
| 202 | + System.out.printf("Usage: number of prompt token is %d, " |
| 203 | + + "number of completion token is %d, and number of total tokens in request and response is %d.%n", |
| 204 | + usage.getPromptTokens(), usage.getCompletionTokens(), usage.getTotalTokens()); |
| 205 | + } |
| 206 | +}); |
| 207 | +``` |
| 208 | + |
| 209 | +### Text embeddings |
| 210 | + |
| 211 | +```java readme-sample-getEmbedding |
| 212 | +EmbeddingsOptions embeddingsOptions = new EmbeddingsOptions( |
| 213 | + Arrays.asList("Your text string goes here")); |
| 214 | + |
| 215 | +Embeddings embeddings = client.getEmbeddings("{deploymentOrModelId}", embeddingsOptions); |
| 216 | + |
| 217 | +for (EmbeddingItem item : embeddings.getData()) { |
| 218 | + System.out.printf("Index: %d.%n", item.getIndex()); |
| 219 | + for (Double embedding : item.getEmbedding()) { |
| 220 | + System.out.printf("%f;", embedding); |
| 221 | + } |
| 222 | +} |
| 223 | +``` |
| 224 | +Please refer to the service documentation for a conceptual discussion of [openAI embedding][microsoft_docs_openai_embedding]. |
| 225 | + |
| 226 | +## Troubleshooting |
| 227 | +### Enable client logging |
| 228 | +You can set the `AZURE_LOG_LEVEL` environment variable to view logging statements made in the client library. For |
| 229 | +example, setting `AZURE_LOG_LEVEL=2` would show all informational, warning, and error log messages. The log levels can |
| 230 | +be found here: [log levels][logLevels]. |
| 231 | + |
| 232 | +### Default HTTP Client |
| 233 | +All client libraries by default use the Netty HTTP client. Adding the above dependency will automatically configure |
| 234 | +the client library to use the Netty HTTP client. Configuring or changing the HTTP client is detailed in the |
| 235 | +[HTTP clients wiki](https://github.com/Azure/azure-sdk-for-java/wiki/HTTP-clients). |
| 236 | + |
| 237 | +### Default SSL library |
| 238 | +All client libraries, by default, use the Tomcat-native Boring SSL library to enable native-level performance for SSL |
| 239 | +operations. The Boring SSL library is an uber jar containing native libraries for Linux / macOS / Windows, and provides |
| 240 | +better performance compared to the default SSL implementation within the JDK. For more information, including how to |
| 241 | +reduce the dependency size, refer to the [performance tuning][performance_tuning] section of the wiki. |
| 242 | + |
| 243 | +## Next steps |
| 244 | +- Samples are explained in detail [here][samples_readme]. |
| 245 | + |
| 246 | +## Contributing |
| 247 | + |
| 248 | +For details on contributing to this repository, see the [contributing guide](https://github.com/Azure/azure-sdk-for-java/blob/main/CONTRIBUTING.md). |
| 249 | + |
| 250 | +1. Fork it |
| 251 | +1. Create your feature branch (`git checkout -b my-new-feature`) |
| 252 | +1. Commit your changes (`git commit -am 'Add some feature'`) |
| 253 | +1. Push to the branch (`git push origin my-new-feature`) |
| 254 | +1. Create new Pull Request |
| 255 | + |
| 256 | +<!-- LINKS --> |
| 257 | +[aad_authorization]: https://docs.microsoft.com/azure/cognitive-services/authentication#authenticate-with-azure-active-directory |
| 258 | +[azure_identity]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/identity/azure-identity |
| 259 | +[azure_identity_credential_type]: https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/identity/azure-identity#credentials |
| 260 | +[azure_openai_access]: https://learn.microsoft.com/azure/cognitive-services/openai/overview#how-do-i-get-access-to-azure-openai |
| 261 | +[azure_subscription]: https://azure.microsoft.com/free/ |
| 262 | +[docs]: https://azure.github.io/azure-sdk-for-java/ |
| 263 | +[jdk]: https://docs.microsoft.com/java/azure/jdk/ |
| 264 | +[logLevels]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/core/azure-core/src/main/java/com/azure/core/util/logging/ClientLogger.java |
| 265 | +[microsoft_docs_openai_completion]: https://learn.microsoft.com/azure/cognitive-services/openai/how-to/completions |
| 266 | +[microsoft_docs_openai_embedding]: https://learn.microsoft.com/azure/cognitive-services/openai/concepts/understand-embeddings |
| 267 | +[performance_tuning]: https://github.com/Azure/azure-sdk-for-java/wiki/Performance-Tuning |
| 268 | +[product_documentation]: https://azure.microsoft.com/services/ |
| 269 | +[quickstart]: https://learn.microsoft.com/azure/cognitive-services/openai/quickstart |
| 270 | +[source_code]: https://github.com/Azure/azure-sdk-for-java/tree/feature/open-ai/sdk/openai/azure-ai-openai/src |
| 271 | +[samples_readme]: https://github.com/Azure/azure-sdk-for-java/tree/feature/open-ai/sdk/openai/azure-ai-openai/src/samples |
| 272 | +[openai_client_async]: https://github.com/Azure/azure-sdk-for-java/blob/feature/open-ai/sdk/openai/azure-ai-openai/src/main/java/com/azure/ai/openai/OpenAIAsyncClient.java |
| 273 | +[openai_client_builder]: https://github.com/Azure/azure-sdk-for-java/blob/feature/open-ai/sdk/openai/azure-ai-openai/src/main/java/com/azure/ai/openai/OpenAIClientBuilder.java |
| 274 | +[openai_client_sync]: https://github.com/Azure/azure-sdk-for-java/blob/feature/open-ai/sdk/openai/azure-ai-openai/src/main/java/com/azure/ai/openai/OpenAIClient.java |
| 275 | +[wiki_identity]: https://github.com/Azure/azure-sdk-for-java/wiki/Identity-and-Authentication |
| 276 | + |
0 commit comments