Merge branch 'main' of https://github.com/MicrosoftDocs/azure-docs-pr into fixVHDScript

Author: roygara

.openpublishing.publish.config.json

@@ -176,6 +176,12 @@
       "branch": "master",
       "branch_mapping": {}
     },
+    {
+      "path_to_root": "azure_powershell_scripts",
+      "url": "https://github.com/Azure-Samples/azure-docs-powershell-samples",
+      "branch": "main",
+      "branch_mapping": {}
+    },
     {
       "path_to_root": "powershell_scripts",
       "url": "https://github.com/Azure/azure-docs-powershell-samples",

.openpublishing.redirection.azure-monitor.json

@@ -6433,6 +6433,21 @@
             "source_path_from_root": "/articles/azure-monitor/faq.yml",
             "redirect_url": "/azure/azure-monitor/overview#frequently-asked-questions",
             "redirect_document_id": false
+        },
+        {
+            "source_path_from_root": "/articles/azure-monitor/logs/app-expression.md",
+            "redirect_url": "/azure/azure-monitor/logs/cross-workspace-query",
+            "redirect_document_id": false
+        },
+        {
+            "source_path_from_root": "/articles/azure-monitor/logs/workspace-expression.md",
+            "redirect_url": "/azure/azure-monitor/logs/cross-workspace-query",
+            "redirect_document_id": false
+        },
+        {
+            "source_path_from_root": "/articles/azure-monitor/logs/resource-expression.md",
+            "redirect_url": "/azure/azure-monitor/logs/cross-workspace-query",
+            "redirect_document_id": false
         }
     ]
 }

.openpublishing.redirection.sentinel.json

@@ -479,6 +479,11 @@
             "source_path": "articles/sentinel/data-connectors/deprecated-cisco-firepower-estreamer-via-legacy-agent.md",
             "redirect_url": "/azure/sentinel/data-connectors/cisco-firepower-estreamer",
             "redirect_document_id": true
-          } 
+          },
+          {
+            "source_path": "articles/sentinel/data-connectors/cyberpion-security-logs.md",
+            "redirect_url": "/azure/sentinel/data-connectors-reference",
+            "redirect_document_id": false
+          }  
     ]
 }

articles/active-directory-b2c/localization-string-ids.md

@@ -8,7 +8,7 @@ manager: CelesteDG
 ms.service: active-directory
 
 ms.topic: reference
-ms.date: 04/19/2022
+ms.date: 11/14/2023
 ms.author: kengaderdus
 ms.subservice: B2C
 ---
@@ -615,7 +615,8 @@ The following IDs are used for claims transformations error messages:
 | `DateTimeGreaterThan` |[AssertDateTimeIsGreaterThan](date-transformations.md#assertdatetimeisgreaterthan) | Claim value comparison failed: The provided left operand is greater than the right operand.|
 | `UserMessageIfClaimsTransformationStringsAreNotEqual` |[AssertStringClaimsAreEqual](string-transformations.md#assertstringclaimsareequal) | Claim value comparison failed using StringComparison "OrdinalIgnoreCase".|
 
-### Claims transformations example
+### Claims transformations example 1:
+This example shows localized messages for local account signup.
 
 ```xml
 <LocalizedResources Id="api.localaccountsignup.en">
@@ -627,6 +628,17 @@ The following IDs are used for claims transformations error messages:
 </LocalizedResources>
 ```
 
+### Claims transformations example 2:
+This example shows localized messages for local account password reset.
+
+```xml
+<LocalizedResources Id="api.localaccountpasswordreset.en">
+  <LocalizedStrings>
+    <LocalizedString ElementType="ErrorMessage" StringId="UserMessageIfClaimsTransformationBooleanValueIsNotEqual">You cannot use the old password</LocalizedString>    
+  </LocalizedStrings>
+</LocalizedResources>
+```
+
 ## Next steps
 
 See the following articles for localization examples:

articles/advisor/advisor-cost-recommendations.md

@@ -5,7 +5,7 @@ author: mabrahms
 ms.author: v-mabrahms
 ms.service: azure
 ms.topic: article
-ms.date: 09/16/2023
+ms.date: 12/1/2023
 
 ---
 
@@ -88,9 +88,9 @@ From any Azure Advisor page, click **Configuration** in the left navigation pane
 
 * **Resources**: Uncheck any subscriptions you don't want to receive Advisor recommendations for, click **Apply**. The page refreshes.
 
-* **VM/VMSS right sizing**: You can adjust the average CPU utilization rule and the look back period on a per-subscription basis. Doing virtual machine (VM) right sizing requires specialized knowledge.
+* **VM/VMSS right sizing**: You can adjust Advisor virtual machine (VM) and virtual machine scale sets (VMSS) recommendations. Specifically, you can setup a filter for each subscription to only show recommendations for machines with certain CPU utilization. This setting will filter recommendations but will not change how they are generated.
 
-  1. Select the subscriptions you’d like to adjust the average CPU utilization rule for, and then click **Edit**. Not all subscriptions can be edited for VM/VMSS right sizing and certain privileges are required; for more information on permissions, see [Permissions in Azure Advisor](permissions.md).
+  1. Select the subscriptions you’d like to setup a filter for average CPU utilization, and then click **Edit**. Not all subscriptions can be edited for VM/VMSS right sizing and certain privileges are required; for more information on permissions, see [Permissions in Azure Advisor](permissions.md).
 
   1. Select the desired average CPU utilization value and click **Apply**. It can take up to 24 hours for the new settings to be reflected in recommendations.
 

articles/advisor/advisor-get-started.md

@@ -18,7 +18,7 @@ items:
       - name: Changelog and release history
         displayName: latest, update, beta, package, preview, version
         href: changelog-release-history.md
-      - name: "SDK targets: REST API 2023–10–31-preview"
+      - name: "🆕 SDK targets: REST API 2023–10–31-preview"
         displayName: get started, installation, downloads, documentAnalysisClient, document analysis client, Azure AD, Azure Active Directory, identity, changelog, package, version,AzureKeyCredential, Azure key credential, key, endpoint
         href: sdk-overview-v4-0.md
       - name: "SDK targets: REST API 2023–7–31 latest (GA)"
@@ -107,20 +107,6 @@ items:
   - name: Back up and recover models
     displayName: disaster, recovery, region, copy, modelId, model ID
     href: disaster-recovery.md
-  - name: Use containers
-    items:
-    - name: Install and run containers
-      displayName: docker, docker-compose, container, compose, packages, az, acr, cli, import, pull, Azure AI Vision, kubernetes, cpu, EULA, billing
-      href: containers/install-run.md
-    - name: Configure containers
-      displayName: docker, docker-compose, compose, containers, run, billing, keys, logging, EULA, telemetry, proxy
-      href: containers/configuration.md
-    - name: Container image tags
-      displayName: registry, mcr
-      href: containers/image-tags.md
-    - name: Disconnected containers
-      displayName: docker, commitment, plan, environment, usage
-      href: containers/disconnected.md
   - name: Configure secure communications
     items:
     - name: Set up Virtual Networks
@@ -138,6 +124,20 @@ items:
     - name: Use Microsoft Entra authentication
       displayName: headers, subscription, access token, azure active directory, subdomain, role, service principal
       href: ../../ai-services/authentication.md?context=/azure/ai-services/document-intelligence/context/context
+- name: Containers
+  items:
+  - name: Install and run containers
+    displayName: docker, docker-compose, container, compose, packages, az, acr, cli, import, pull, Azure AI Vision, kubernetes, cpu, EULA, billing
+    href: containers/install-run.md
+  - name: Configure containers
+    displayName: docker, docker-compose, compose, containers, run, billing, keys, logging, EULA, telemetry, proxy
+    href: containers/configuration.md
+  - name: Container image tags
+    displayName: registry, mcr
+    href: containers/image-tags.md
+  - name: Disconnected containers
+    displayName: docker, commitment, plan, environment, usage
+    href: containers/disconnected.md
 - name: Samples
   items:
   - name: "C#/.NET"
@@ -244,7 +244,6 @@ items:
     displayName: connectors, automate, automation, workflows
     href: tutorial-logic-apps.md
   - name: "Create Document Intelligence workflows with AI Builder"
-    expanded: true
     items:
     - name:  Custom
       href: /ai-builder/create-form-processing-model?toc=/azure/ai-services/document-intelligence/toc.json&bc=/azure/ai-services/document-intelligence/breadcrumb/toc.json

articles/ai-services/document-intelligence/toc.yml

@@ -237,26 +237,15 @@ To add a new data source to Azure OpenAI on your data, you need the following Az
 | [Contributor](/azure/role-based-access-control/built-in-roles#contributor) | Your subscription, to access Azure Resource Manager. | You want to deploy a web app. |
 | [Cognitive Services Contributor Role](/azure/role-based-access-control/built-in-roles#cognitive-services-contributor) | The Azure AI Search resource, to access Azure OpenAI resource. | You want to deploy a [web app](#using-the-web-app).   |
 
-## Virtual network support & private endpoint support (Azure AI Search only)
+## Virtual network support & private endpoint support
 
-> [!TIP]
-> For instructions on setting up your resources to work on a virtual private network or private endpoint, see [Use Azure OpenAI on your data securely](../how-to/use-your-data-securely.md)
-
-### Azure OpenAI resources
-
-You can protect Azure OpenAI resources in [virtual networks and private endpoints](/azure/ai-services/cognitive-services-virtual-networks) the same way as any Azure AI service.
-
-### Azure AI Search resources
-
-If you have an Azure AI Search resource protected by a private network, and want to allow Azure OpenAI on your data to access your search service, complete [an application form](https://aka.ms/applyacsvpnaoaioyd). The application will be reviewed in ten business days and you will be contacted via email about the results. If you are eligible, we will send a private endpoint request to your search service, and you will need to approve the request.
+* For instructions on setting up your resources to work on a virtual private network or private endpoint, see [Use Azure OpenAI on your data securely](../how-to/use-your-data-securely.md)
+* Azure OpenAI, Azure AI Search, and Azure Storage Accounts can be protected under private endpoints and virtual private networks.
 
-:::image type="content" source="../media/use-your-data/approve-private-endpoint.png" alt-text="A screenshot showing private endpoint approval screen." lightbox="../media/use-your-data/approve-private-endpoint.png":::
+## Document-level access control
 
-Learn more about the [manual approval workflow](/azure/private-link/private-endpoint-overview#access-to-a-private-link-resource-using-approval-workflow).
-
-After you approve the request in your search service, you can start using the [chat completions extensions API](/azure/ai-services/openai/reference#completions-extensions). Public network access can be disabled for that search service.
-
-## Document-level access control (Azure AI Search only)
+> [!NOTE] 
+> Document-level access control is supported for Azure AI search only.
 
 Azure OpenAI on your data lets you restrict the documents that can be used in responses for different users with Azure AI Search [security filters](/azure/search/search-security-trimming-for-azure-search-with-aad). When you enable document level access, the search results returned from Azure AI Search and used to generate a response will be trimmed based on user Microsoft Entra group membership. You can only enable document-level access on existing Azure AI Search indexes. To enable document-level access:
 
@@ -308,7 +297,10 @@ When using the API, pass the `filter` parameter in each API request. For example
 * `my_group_ids` is the field name that you selected for **Permitted groups** during [fields mapping](#index-field-mapping).
 * `group_id1, group_id2` are groups attributed to the logged in user. The client application can retrieve and cache users' groups.
 
-## Schedule automatic index refreshes (Azure AI Search only)
+## Schedule automatic index refreshes
+
+> [!NOTE] 
+> Automatic index refreshing is supported for Azure Blob storage only.
 
 To keep your Azure AI Search index up-to-date with your latest data, you can schedule a refresh for it that runs automatically rather than manually updating it every time your data is updated. Automatic index refresh is only available when you choose **blob storage** as the data source. To enable an automatic index refresh:
 
@@ -542,6 +534,37 @@ When you chat with a model, providing a history of the chat will help the model
 }
 ```
 
+## Token usage estimation for Azure OpenAI on your data
+
+
+| Model                   | Total tokens available | Max tokens for system message | Max tokens for model response |
+|-------------------------|------------------------|------------------------------------|------------------------------------|
+| ChatGPT Turbo (0301) 8k | 8000                   | 400                                | 1500                               |
+| ChatGPT Turbo 16k       | 16000                  | 1000                               | 3200                               |
+| GPT-4 (8k)              | 8000                   | 400                                | 1500                               |
+| GPT-4 32k               | 32000                  | 2000                               | 6400                               |
+
+The table above shows the total number of tokens available for each model type. It also determines the maximum number of tokens that can be used for the [system message](#system-message) and the model response. Additionally, the following also consume tokens:
+
+
+* The meta prompt (MP): if you limit responses from the model to the grounding data content (`inScope=True` in the API), the maximum number of tokens is 4036 tokens. Otherwise (for example if `inScope=False`) the maximum is 3444 tokens. This number is variable depending on the token length of the user question and conversation history. This estimate includes the base prompt as well as the query rewriting prompts for retrieval.
+* User question and history: Variable but capped at 2000 tokens.
+* Retrieved documents (chunks): The number of tokens used by the retrieved document chunks depends on multiple factors. The upper bound for this is the number of retrieved document chunks multiplied by the chunk size. It will, however, be truncated based on the tokens available tokens for the specific model being used after counting the rest of fields. 
+
+    20% of the available tokens are reserved for the model response. The remaining 80% of available tokens include the meta prompt, the user question and conversation history, and the system message. The remaining token budget is used by the retrieved document chunks. 
+
+```python 
+import tiktoken
+
+class TokenEstimator(object):
+
+    GPT2_TOKENIZER = tiktoken.get_encoding("gpt2")
+
+    def estimate_tokens(self, text: str) -> int:
+        return len(self.GPT2_TOKENIZER.encode(text))
+      
+token_output = TokenEstimator.estimate_tokens(input_text)
+```
 
 ## Next steps
 * [Get started using your data with Azure OpenAI](../use-your-data-quickstart.md)

articles/ai-services/openai/concepts/use-your-data.md

@@ -212,7 +212,7 @@ sections:
       - question: | 
           How can I update my index?
         answer:
-          You can [schedule an automatic index refresh](concepts/use-your-data.md#schedule-automatic-index-refreshes-azure-ai-search-only), or upload additional data to your Azure Blob Container and use it as your data source when you create a new index. The new index will include all of the data in your container.
+          You can [schedule an automatic index refresh](concepts/use-your-data.md#schedule-automatic-index-refreshes), or upload additional data to your Azure Blob Container and use it as your data source when you create a new index. The new index will include all of the data in your container.
       - question: | 
           What file types does Azure OpenAI on your data support?
         answer:

articles/ai-services/openai/faq.yml

@@ -52,7 +52,7 @@ client = AzureOpenAI(
     api_key=os.getenv("AZURE_OPENAI_KEY"),  
     api_version="2023-10-01-preview",
     azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT")
-    )
+)
 ```
 
 </td>