Merge pull request #260078 from MicrosoftDocs/main

12/1/2023 PM Publish

Author: Taojunshen

.openpublishing.publish.config.json

@@ -176,6 +176,12 @@
       "branch": "master",
       "branch_mapping": {}
     },
+    {
+      "path_to_root": "azure_powershell_scripts",
+      "url": "https://github.com/Azure-Samples/azure-docs-powershell-samples",
+      "branch": "main",
+      "branch_mapping": {}
+    },
     {
       "path_to_root": "powershell_scripts",
       "url": "https://github.com/Azure/azure-docs-powershell-samples",

articles/ai-services/document-intelligence/toc.yml

@@ -18,7 +18,7 @@ items:
       - name: Changelog and release history
         displayName: latest, update, beta, package, preview, version
         href: changelog-release-history.md
-      - name: "SDK targets: REST API 2023–10–31-preview"
+      - name: "🆕 SDK targets: REST API 2023–10–31-preview"
         displayName: get started, installation, downloads, documentAnalysisClient, document analysis client, Azure AD, Azure Active Directory, identity, changelog, package, version,AzureKeyCredential, Azure key credential, key, endpoint
         href: sdk-overview-v4-0.md
       - name: "SDK targets: REST API 2023–7–31 latest (GA)"
@@ -107,20 +107,6 @@ items:
   - name: Back up and recover models
     displayName: disaster, recovery, region, copy, modelId, model ID
     href: disaster-recovery.md
-  - name: Use containers
-    items:
-    - name: Install and run containers
-      displayName: docker, docker-compose, container, compose, packages, az, acr, cli, import, pull, Azure AI Vision, kubernetes, cpu, EULA, billing
-      href: containers/install-run.md
-    - name: Configure containers
-      displayName: docker, docker-compose, compose, containers, run, billing, keys, logging, EULA, telemetry, proxy
-      href: containers/configuration.md
-    - name: Container image tags
-      displayName: registry, mcr
-      href: containers/image-tags.md
-    - name: Disconnected containers
-      displayName: docker, commitment, plan, environment, usage
-      href: containers/disconnected.md
   - name: Configure secure communications
     items:
     - name: Set up Virtual Networks
@@ -138,6 +124,20 @@ items:
     - name: Use Microsoft Entra authentication
       displayName: headers, subscription, access token, azure active directory, subdomain, role, service principal
       href: ../../ai-services/authentication.md?context=/azure/ai-services/document-intelligence/context/context
+- name: Containers
+  items:
+  - name: Install and run containers
+    displayName: docker, docker-compose, container, compose, packages, az, acr, cli, import, pull, Azure AI Vision, kubernetes, cpu, EULA, billing
+    href: containers/install-run.md
+  - name: Configure containers
+    displayName: docker, docker-compose, compose, containers, run, billing, keys, logging, EULA, telemetry, proxy
+    href: containers/configuration.md
+  - name: Container image tags
+    displayName: registry, mcr
+    href: containers/image-tags.md
+  - name: Disconnected containers
+    displayName: docker, commitment, plan, environment, usage
+    href: containers/disconnected.md
 - name: Samples
   items:
   - name: "C#/.NET"
@@ -244,7 +244,6 @@ items:
     displayName: connectors, automate, automation, workflows
     href: tutorial-logic-apps.md
   - name: "Create Document Intelligence workflows with AI Builder"
-    expanded: true
     items:
     - name:  Custom
       href: /ai-builder/create-form-processing-model?toc=/azure/ai-services/document-intelligence/toc.json&bc=/azure/ai-services/document-intelligence/breadcrumb/toc.json

articles/ai-services/openai/concepts/use-your-data.md

@@ -237,26 +237,15 @@ To add a new data source to Azure OpenAI on your data, you need the following Az
 | [Contributor](/azure/role-based-access-control/built-in-roles#contributor) | Your subscription, to access Azure Resource Manager. | You want to deploy a web app. |
 | [Cognitive Services Contributor Role](/azure/role-based-access-control/built-in-roles#cognitive-services-contributor) | The Azure AI Search resource, to access Azure OpenAI resource. | You want to deploy a [web app](#using-the-web-app).   |
 
-## Virtual network support & private endpoint support (Azure AI Search only)
+## Virtual network support & private endpoint support
 
-> [!TIP]
-> For instructions on setting up your resources to work on a virtual private network or private endpoint, see [Use Azure OpenAI on your data securely](../how-to/use-your-data-securely.md)
-
-### Azure OpenAI resources
-
-You can protect Azure OpenAI resources in [virtual networks and private endpoints](/azure/ai-services/cognitive-services-virtual-networks) the same way as any Azure AI service.
-
-### Azure AI Search resources
-
-If you have an Azure AI Search resource protected by a private network, and want to allow Azure OpenAI on your data to access your search service, complete [an application form](https://aka.ms/applyacsvpnaoaioyd). The application will be reviewed in ten business days and you will be contacted via email about the results. If you are eligible, we will send a private endpoint request to your search service, and you will need to approve the request.
+* For instructions on setting up your resources to work on a virtual private network or private endpoint, see [Use Azure OpenAI on your data securely](../how-to/use-your-data-securely.md)
+* Azure OpenAI, Azure AI Search, and Azure Storage Accounts can be protected under private endpoints and virtual private networks.
 
-:::image type="content" source="../media/use-your-data/approve-private-endpoint.png" alt-text="A screenshot showing private endpoint approval screen." lightbox="../media/use-your-data/approve-private-endpoint.png":::
+## Document-level access control
 
-Learn more about the [manual approval workflow](/azure/private-link/private-endpoint-overview#access-to-a-private-link-resource-using-approval-workflow).
-
-After you approve the request in your search service, you can start using the [chat completions extensions API](/azure/ai-services/openai/reference#completions-extensions). Public network access can be disabled for that search service.
-
-## Document-level access control (Azure AI Search only)
+> [!NOTE] 
+> Document-level access control is supported for Azure AI search only.
 
 Azure OpenAI on your data lets you restrict the documents that can be used in responses for different users with Azure AI Search [security filters](/azure/search/search-security-trimming-for-azure-search-with-aad). When you enable document level access, the search results returned from Azure AI Search and used to generate a response will be trimmed based on user Microsoft Entra group membership. You can only enable document-level access on existing Azure AI Search indexes. To enable document-level access:
 
@@ -308,7 +297,10 @@ When using the API, pass the `filter` parameter in each API request. For example
 * `my_group_ids` is the field name that you selected for **Permitted groups** during [fields mapping](#index-field-mapping).
 * `group_id1, group_id2` are groups attributed to the logged in user. The client application can retrieve and cache users' groups.
 
-## Schedule automatic index refreshes (Azure AI Search only)
+## Schedule automatic index refreshes
+
+> [!NOTE] 
+> Automatic index refreshing is supported for Azure Blob storage only.
 
 To keep your Azure AI Search index up-to-date with your latest data, you can schedule a refresh for it that runs automatically rather than manually updating it every time your data is updated. Automatic index refresh is only available when you choose **blob storage** as the data source. To enable an automatic index refresh:
 
@@ -542,6 +534,37 @@ When you chat with a model, providing a history of the chat will help the model
 }
 ```
 
+## Token usage estimation for Azure OpenAI on your data
+
+
+| Model                   | Total tokens available | Max tokens for system message | Max tokens for model response |
+|-------------------------|------------------------|------------------------------------|------------------------------------|
+| ChatGPT Turbo (0301) 8k | 8000                   | 400                                | 1500                               |
+| ChatGPT Turbo 16k       | 16000                  | 1000                               | 3200                               |
+| GPT-4 (8k)              | 8000                   | 400                                | 1500                               |
+| GPT-4 32k               | 32000                  | 2000                               | 6400                               |
+
+The table above shows the total number of tokens available for each model type. It also determines the maximum number of tokens that can be used for the [system message](#system-message) and the model response. Additionally, the following also consume tokens:
+
+
+* The meta prompt (MP): if you limit responses from the model to the grounding data content (`inScope=True` in the API), the maximum number of tokens is 4036 tokens. Otherwise (for example if `inScope=False`) the maximum is 3444 tokens. This number is variable depending on the token length of the user question and conversation history. This estimate includes the base prompt as well as the query rewriting prompts for retrieval.
+* User question and history: Variable but capped at 2000 tokens.
+* Retrieved documents (chunks): The number of tokens used by the retrieved document chunks depends on multiple factors. The upper bound for this is the number of retrieved document chunks multiplied by the chunk size. It will, however, be truncated based on the tokens available tokens for the specific model being used after counting the rest of fields. 
+
+    20% of the available tokens are reserved for the model response. The remaining 80% of available tokens include the meta prompt, the user question and conversation history, and the system message. The remaining token budget is used by the retrieved document chunks. 
+
+```python 
+import tiktoken
+
+class TokenEstimator(object):
+
+    GPT2_TOKENIZER = tiktoken.get_encoding("gpt2")
+
+    def estimate_tokens(self, text: str) -> int:
+        return len(self.GPT2_TOKENIZER.encode(text))
+      
+token_output = TokenEstimator.estimate_tokens(input_text)
+```
 
 ## Next steps
 * [Get started using your data with Azure OpenAI](../use-your-data-quickstart.md)

articles/ai-services/openai/faq.yml

@@ -212,7 +212,7 @@ sections:
       - question: | 
           How can I update my index?
         answer:
-          You can [schedule an automatic index refresh](concepts/use-your-data.md#schedule-automatic-index-refreshes-azure-ai-search-only), or upload additional data to your Azure Blob Container and use it as your data source when you create a new index. The new index will include all of the data in your container.
+          You can [schedule an automatic index refresh](concepts/use-your-data.md#schedule-automatic-index-refreshes), or upload additional data to your Azure Blob Container and use it as your data source when you create a new index. The new index will include all of the data in your container.
       - question: | 
           What file types does Azure OpenAI on your data support?
         answer:

articles/ai-services/openai/how-to/use-your-data-securely.md

@@ -27,7 +27,7 @@ When you ingest data into Azure OpenAI on your data, the following process is us
 
 1. The ingestion process is started when a client sends data to be processed.
 1. Ingestion assets (indexers, indexes, data sources, a [custom skill](/azure/search/cognitive-search-custom-skill-interface) and container in the search resource) are created in the Azure AI Search resource and Azure storage account.
-1. If the ingestion is triggered by a [scheduled refresh](../concepts/use-your-data.md#schedule-automatic-index-refreshes-azure-ai-search-only), the ingestion process starts at `[3]`.
+1. If the ingestion is triggered by a [scheduled refresh](../concepts/use-your-data.md#schedule-automatic-index-refreshes), the ingestion process starts at `[3]`.
 1.  Azure OpenAI's `preprocessing-jobs` API implements the [Azure AI Search customer skill web API protocol](/azure/search/cognitive-search-custom-skill-web-api), and processes the documents in a queue. 
 1. Azure OpenAI:
     1. Internally uses the indexer created earlier to crack the documents.
@@ -110,6 +110,8 @@ To set the managed identities via the management API, see [the management API re
 
 ## Security support for Azure AI Search
 
+You can protect Azure OpenAI resources in [virtual networks and private endpoints](/azure/ai-services/cognitive-services-virtual-networks) the same way as any Azure AI service.
+
 ### Inbound security: authentication
 As Azure OpenAI will use managed identity to access Azure AI Search, you need to enable Azure AD based authentication in your Azure AI Search. To do it on Azure portal, select **Both** in the **Keys** tab in the Azure portal.
 
@@ -133,7 +135,11 @@ To use Azure OpenAI Studio, you can't disable the API key based authentication f
 
 ### Inbound security: networking
 
-Use **Selected networks** in the Azure portal. Azure AI Search doesn't support bypassing trusted services, so it is the most complex part in the setup. Create a private endpoint for theAzure OpenAI on your data (as a multitenant service managed by Microsoft), and link it to your Azure AI Search resource. This requires you to submit an [application form](https://aka.ms/applyacsvpnaoaioyd).
+Use **Selected networks** in the Azure portal. Azure AI Search doesn't support bypassing trusted services, so it is the most complex part in the setup. Create a private endpoint for the Azure OpenAI on your data resource (as a multitenant service managed by Microsoft), and link it to your Azure AI Search resource. This requires you to submit an [application form](https://aka.ms/applyacsvpnaoaioyd). The application will be reviewed in ten business days and you will be contacted via email about the results. If you are eligible, we will send a private endpoint request to your search service, and you will need to approve the request.
+
+:::image type="content" source="../media/use-your-data/approve-private-endpoint.png" alt-text="A screenshot showing private endpoint approval screen." lightbox="../media/use-your-data/approve-private-endpoint.png":::
+
+Learn more about the [manual approval workflow](/azure/private-link/private-endpoint-overview#access-to-a-private-link-resource-using-approval-workflow).
 
 > [!NOTE]
 > To use Azure OpenAI Studio, you cannot disable public network access, and you need to add your local IP to the IP rules, because Azure AI Studio calls the search API from your browser to list available indexes.