suggested changes

Author: aahill

articles/ai-services/openai/concepts/assistants.md

@@ -1,5 +1,5 @@
 ---
-title: Azure OpenAI Service Assistant API concepts
+title: Azure OpenAI Service Assistants API concepts
 titleSuffix: Azure OpenAI Service
 description: Learn about the concepts behind the Azure OpenAI Assistants API.
 ms.topic: conceptual
@@ -15,6 +15,11 @@ recommendations: false
 
 Assistants, a new feature of Azure OpenAI Service, is now available in public preview. Assistants API makes it easier for developers to create applications with sophisticated copilot-like experiences that can sift through data, suggest solutions, and automate tasks.
 
+* Assistants can call Azure OpenAI’s [models](../concepts/models.md) with specific instructions to tune their personality and capabilities.
+* Assistants can access **multiple tools in parallel**. These can be both Azure OpenAI-hosted tools like [code interpreter](../how-to/code-interpreter.md) and [file search](../how-to/file-search.md), or tools you build, host, and access through [function calling](../how-to/function-calling.md).
+* Assistants can access **persistent Threads**. Threads simplify AI application development by storing message history and truncating it when the conversation gets too long for the model's context length. You create a Thread once, and simply append Messages to it as your users reply.
+* Assistants can access files in several formats. Either as part of their creation or as part of Threads between Assistants and users. When using tools, Assistants can also create files (such as images or spreadsheets) and cite files they reference in the Messages they create.
+
 ## Overview
 
 Previously, building custom AI assistants needed heavy lifting even for experienced developers. While the chat completions API is lightweight and powerful, it's inherently stateless, which means that developers had to manage conversation state and chat threads, tool integrations, retrieval documents and indexes, and execute code manually.
@@ -26,19 +31,21 @@ Assistants API supports persistent automatically managed threads. This means tha
 - [Function calling](../how-to/assistant-functions.md)
 
 > [!TIP]
-> There is no additional [pricing](https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/) or [quota](../quotas-limits.md) for using Assistants unless you use the [code interpreter](../how-to/code-interpreter.md) tool.
+> There is no additional [pricing](https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/) or [quota](../quotas-limits.md) for using Assistants unless you use the [code interpreter](../how-to/code-interpreter.md) or [file search](../how-to/) tools.
 
-Assistant API is built on the same capabilities that power OpenAI’s GPT product. Some possible use cases range from AI-powered product recommender, sales analyst app, coding assistant, employee Q&A chatbot, and more. Start building on the no-code Assistants playground on the Azure OpenAI Studio or start building with the API.
+Assistants API is built on the same capabilities that power OpenAI’s GPT product. Some possible use cases range from AI-powered product recommender, sales analyst app, coding assistant, employee Q&A chatbot, and more. Start building on the no-code Assistants playground on the Azure OpenAI Studio, AI Studio, or start building with the API.
 
 > [!IMPORTANT]
-> Retrieving untrusted data using Function calling, Code Interpreter with file input, and Assistant Threads functionalities could compromise the security of your Assistant, or the application that uses the Assistant. Learn about mitigation approaches [here](https://aka.ms/oai/assistant-rai).
+> Retrieving untrusted data using Function calling, Code Interpreter or File Search with file input, and Assistant Threads functionalities could compromise the security of your Assistant, or the application that uses the Assistant. Learn about mitigation approaches [here](https://aka.ms/oai/assistant-rai).
 
 ## Assistants playground
 
 We provide a walkthrough of the Assistants playground in our [quickstart guide](../assistants-quickstart.md). This provides a no-code environment to test out the capabilities of assistants.
 
 ## Assistants components
 
+:::image type="content" source="../media/assistants/assistants-overview.png" alt-text="A diagram showing the components of an assistant." lightbox="../media/assistants/assistants-overview.png:::
+
 | **Component** | **Description** |
 |---|---|
 | **Assistant** | Custom AI that uses Azure OpenAI models in conjunction with tools. |
@@ -77,7 +84,7 @@ When using the File Search tool, we recommend setting the `max_prompt_tokens` to
 You may also specify a truncation strategy to control how your thread should be rendered into the model's context window. Using a truncation strategy of type `auto` will use OpenAI's default truncation strategy. Using a truncation strategy of type `last_messages` will allow you to specify the number of the most recent messages to include in the context window.
 
 ## See also
-
+* Learn more about Assistants and [File Search](../how-to/file-search.md)
 * Learn more about Assistants and [Code Interpreter](../how-to/code-interpreter.md)
 * Learn more about Assistants and [function calling](../how-to/assistant-functions.md)
 * [Azure OpenAI Assistants API samples](https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/Assistants)

articles/ai-services/openai/how-to/assistant-functions.md

@@ -33,6 +33,10 @@ To use all features of function calling including parallel functions, you need t
 
 ## Example function definition
 
+> [!NOTE]
+> * We've added support for the `tool_choice` parameter which can be used to force the use of a specific tool (like `file_search`, `code_interpreter`, or a `function`) in a particular run. 
+> * Runs expire ten minutes after creation. Be sure to submit your tool outputs before the 10 min mark.
+
 # [Python 1.x](#tab/python)
 
 ```python

articles/ai-services/openai/how-to/assistant.md

@@ -24,7 +24,7 @@ Azure OpenAI Assistants (Preview) allows you to create AI assistants tailored to
 
 ### Region and model support
 
-The [models page](../concepts/models.md#assistants-preview) contains the most up-to-date information on regions/models where Assistants are currently supported. Some [tools](#tools) may have limited regional or model support. For more information, see the article for the tools you want to use.
+Code interpreter is available in all regions supported by Azure OpenAI Assistants. The [models page](../concepts/models.md#assistants-preview) contains the most up-to-date information on regions/models where Assistants are currently supported.
 
 ### API Version
 
@@ -64,6 +64,9 @@ The [models page](../concepts/models.md#assistants-preview) contains the most up
 
 ### Tools
 
+> [!TIP]
+> We've added support for the `tool_choice` parameter which can be used to force the use of a specific tool (like `file_search`, `code_interpreter`, or a `function`) in a particular run. 
+
 An individual assistant can access up to 128 tools including [code interpreter](./code-interpreter.md) and [file search](./file-search.md), but you can also define your own custom tools via [functions](./assistant-functions.md).
 
 ### Files
@@ -120,9 +123,9 @@ assistant = client.beta.assistants.create(
 
 There are a few details you should note from the configuration above:
 
-- We enable this assistant to access code interpreter with the line ` tools=[{"type": "code_interpreter"}],`. This gives the model access to a sand-boxed python environment to run and execute code to help formulating responses to a user's question.
-- In the instructions we remind the model that it can execute code. Sometimes the model needs help guiding it towards the right tool to solve a given query. If you know, you want to use a particular library to generate a certain response that you know is part of code interpreter it can help to provide guidance by saying something like "Use Matplotlib to do x."
-- Since this is Azure OpenAI the value you enter for `model=` **must match the deployment name**. By convention our docs will often use a deployment name that happens to match the model name to indicate which model was used when testing a given example, but in your environment the deployment names can be different and that is the name that you should enter in the code.
+- We enable this assistant to access code interpreter with the line `tools=[{"type": "code_interpreter"}],`. This gives the model access to a sand-boxed python environment to run and execute code to help formulating responses to a user's question.
+- In the instructions we remind the model that it can execute code. Sometimes the model needs help guiding it towards the right tool to solve a given query. If you know you want to use a particular library to generate a certain response that you know is part of code interpreter, it can help to provide guidance by saying something like "Use Matplotlib to do x."
+- Since this is Azure OpenAI the value you enter for `model=` **must match the deployment name**.
 
 Next we're going to print the contents of assistant that we just created to confirm that creation was successful:
 

articles/ai-services/openai/how-to/file-search.md

@@ -16,6 +16,11 @@ recommendations: false
 
 File Search augments the Assistant with knowledge from outside its model, such as proprietary product information or documents provided by your users. OpenAI automatically parses and chunks your documents, creates and stores the embeddings, and use both vector and keyword search to retrieve relevant content to answer user queries.
 
+> [!IMPORTANT]
+> File search has [additional charges](https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/) beyond the token based fees for Azure OpenAI usage. If your Assistant calls file search simultaneously in two different threads, two file search sessions are created.
+
+[!INCLUDE [Assistants v2 note](../includes/assistants-v2-note.md)]
+
 ## File search support
 
 ### Supported regions

articles/ai-services/openai/how-to/function-calling.md

@@ -52,6 +52,8 @@ To force the model to call a specific function set the `tool_choice` parameter w
 > [!NOTE]
 > The default behavior (`tool_choice: "auto"`) is for the model to decide on its own whether to call a function and if so which function to call.
 
+#### [Non-streaming](#tab/non-streaming)
+
 ```python
 import os
 from openai import AzureOpenAI
@@ -141,6 +143,66 @@ def run_conversation():
 print(run_conversation())
 ```
 
+#### [Streaming](#tab/streaming)
+
+```python
+import os
+from openai import AzureOpenAI
+import json
+
+client = AzureOpenAI(
+  azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"), 
+  api_key=os.getenv("AZURE_OPENAI_API_KEY"),  
+  api_version="2024-03-01-preview"
+)
+
+from typing_extensions import override
+from openai import AssistantEventHandler
+ 
+class EventHandler(AssistantEventHandler):
+    @override
+    def on_event(self, event):
+      # Retrieve events that are denoted with 'requires_action'
+      # since these will have our tool_calls
+      if event.event == 'thread.run.requires_action':
+        run_id = event.data.id  # Retrieve the run ID from the event data
+        self.handle_requires_action(event.data, run_id)
+ 
+    def handle_requires_action(self, data, run_id):
+      tool_outputs = []
+        
+      for tool in data.required_action.submit_tool_outputs.tool_calls:
+        if tool.function.name == "get_current_temperature":
+          tool_outputs.append({"tool_call_id": tool.id, "output": "57"})
+        elif tool.function.name == "get_rain_probability":
+          tool_outputs.append({"tool_call_id": tool.id, "output": "0.06"})
+        
+      # Submit all tool_outputs at the same time
+      self.submit_tool_outputs(tool_outputs, run_id)
+ 
+    def submit_tool_outputs(self, tool_outputs, run_id):
+      # Use the submit_tool_outputs_stream helper
+      with client.beta.threads.runs.submit_tool_outputs_stream(
+        thread_id=self.current_run.thread_id,
+        run_id=self.current_run.id,
+        tool_outputs=tool_outputs,
+        event_handler=EventHandler(),
+      ) as stream:
+        for text in stream.text_deltas:
+          print(text, end="", flush=True)
+        print()
+ 
+ 
+with client.beta.threads.runs.stream(
+  thread_id=thread.id,
+  assistant_id=assistant.id,
+  event_handler=EventHandler()
+) as stream:
+  stream.until_done()
+```
+
+--- 
+
 ## Using function in the chat completions API (Deprecated)
 
 Function calling is available in the `2023-07-01-preview` API version and works with version 0613 of gpt-35-turbo, gpt-35-turbo-16k, gpt-4, and gpt-4-32k.

articles/ai-services/openai/includes/assistants-v2-note.md

@@ -8,4 +8,6 @@ ms.date: 04/22/2024
 ---
 
 > [!NOTE]
-> Azure OpenAI does not yet support Assistants V2. Please use the v1.20.0 release of the OpenAI Python library until V2 support is available.
+> * [File search](../how-to/file-search.md) can ingest up to 10,000 files per assistant - 500 times more than before. It is fast, supports parallel queries through multi-threaded searches, and features enhanced reranking and query rewriting.
+>     * Vector store is a new object in the API. Once a file is added to a vector store, it's automatically parsed, chunked, and embedded, made ready to be searched. Vector stores can be used across assistants and threads, simplifying file management and billing.
+> * We've added support for the `tool_choice` parameter which can be used to force the use of a specific tool (like file search, code interpreter, or a function) in a particular run.

articles/ai-services/openai/media/assistants/assistants-overview.png

@@ -27,8 +27,11 @@ A refresh of the Assistants API is now publicly available. It contains the follo
 * [File search tool and vector storage](./how-to/file-search.md)
 * [Max completion and max prompt token support](./concepts/assistants.md#context-window-management) for managing token usage.
 * `tool_choice` [parameter](./assistants-reference-runs.md#run-object) for forcing the Assistant to use use a specified tool. 
-* New [assistant role](.//assistants-reference-messages.md#create-message) for creating custom conversation histories. 
+You can now create messages with the [assistant](.//assistants-reference-messages.md#create-message) role to create custom conversation histories in Threads.
 * Support for `temperature`, `top_p`, `response_format` [parameters](./assistants-reference.md#create-an-assistant).
+* Streaming and polling support. You can use the helper functions in our Python SDK to create runs and stream responses. We have also added polling SDK helpers to share object status updates without the need for polling. 
+* Experiment with Logic Apps and Function Calling using Azure OpenAI Studio. Import your REST APIs implemented in Logic Apps as functions and the studio invokes the function (as a Logic Apps workflow) automatically based on the user prompt.
+* AutoGen by Microsoft Research provides a multi-agent conversation framework to enable convenient building of Large Language Model (LLM) workflows across a wide range of applications. Azure OpenAI assistants are now integrated into AutoGen via `GPTAssistantAgent`, a new experimental agent that lets you seamlessly add Assistants into AutoGen-based multi-agent workflows. This enables multiple Azure OpenAI assistants, that could be task or domain specialized, to collaborate and tackle complex tasks.
 * Support for fine-tuned `gpt-3.5-turbo-0125` [models](./concepts/models.md#assistants-preview) in the following regions:
     * East US 2
     * Sweden Central