Merge branch 'MicrosoftDocs:main' into v1-migration

Author: MJyot

articles/ai-services/language-service/conversational-language-understanding/concepts/best-practices.md

@@ -121,6 +121,14 @@ Once the request is sent, you can track the progress of the training job in Lang
 > [!NOTE]
 > You have to retrain your model after updating the `confidenceThreshold` project setting. Afterwards, you'll need to republish the app for the new threshold to take effect.
 
+### Normalization in model version 2023-04-15
+
+Model version 2023-04-15, conversational language understanding provides normalization in the inference layer that doesn't affect training. 
+
+The normalization layer normalizes the classification confidence scores to a confined range. The range selected currently is from `[-a,a]` where "a" is the square root of the number of intents.  As a result, the normalization depends on the number of intents in the app. If there is a very low number of intents, the normalization layer has a very small range to work with. With a fairly large number of intents, the normalization is more effective.
+
+If this normalization doesn’t seem to help intents that are out of scope to the extent that the confidence threshold can be used to filter out of scope utterances, it might be related to the number of intents in the app. Consider adding more intents to the app, or if you are using an orchestrated architecture, consider merging apps that belong to the same domain together. 
+
 ## Debugging composed entities
 
 Entities are functions that emit spans in your input with an associated type. The function is defined by one or more components. You can mark components as needed, and you can decide whether to enable the *combine components* setting. When you combine components, all spans that overlap will be merged into a single span. If the setting isn't used, each individual component span will be emitted.

articles/ai-services/openai/how-to/embeddings.md

@@ -6,9 +6,9 @@ services: cognitive-services
 manager: nitinme
 ms.service: azure-ai-openai
 ms.topic: how-to
-ms.date: 9/12/2023
-author: ChrisHMSFT
-ms.author: chrhoder
+ms.date: 11/02/2023
+author: mrbullwinkle
+ms.author: mbullwin
 recommendations: false
 keywords: 
 
@@ -75,15 +75,7 @@ foreach (float item in returnValue.Value.Data[0].Embedding)
 
 ### Verify inputs don't exceed the maximum length
 
-The maximum length of input text for our embedding models is 2048 tokens (equivalent to around 2-3 pages of text). You should verify that your inputs don't exceed this limit before making a request.
-
-### Choose the best model for your task
-
-For the search models, you can obtain embeddings in two ways. The `<search_model>-doc` model is used for longer pieces of text (to be searched over) and the `<search_model>-query` model is used for shorter pieces of text, typically queries or class labels in zero shot classification. You can read more about all of the Embeddings models in our [Models](../concepts/models.md) guide.
-
-### Replace newlines with a single space
-
-Unless you're embedding code, we suggest replacing newlines (\n) in your input with a single space, as we have observed inferior results when newlines are present.
+The maximum length of input text for our latest embedding models is 8192 tokens. You should verify that your inputs don't exceed this limit before making a request.
 
 ## Limitations & risks
 

articles/app-service/configure-basic-auth-disable.md

@@ -0,0 +1,154 @@
+---
+title: Disable basic authentication for deployment
+description: Learn how to secure App Service deployment by disabling basic authentication.
+keywords: azure app service, security, deployment, FTP, MsDeploy
+ms.topic: article
+ms.date: 11/05/2023
+author: cephalin
+ms.author: cephalin
+---
+
+# Disable basic authentication in App Service deployments
+
+This article shows you how to disable basic authentication (username and password authentication) when deploying code to App Service apps.
+
+App Service provides basic authentication for FTP and WebDeploy clients to connect to it by using [deployment credentials](deploy-configure-credentials.md). These APIs are great for browsing your site’s file system, uploading drivers and utilities, and deploying with MsBuild. However, enterprises often require more secure deployment methods than basic authentication, such as [Microsoft Entra ID](/entra/fundamentals/whatis) authentication (see [Authentication types by deployment methods in Azure App Service](deploy-authentication-types.md)). Entra ID uses OAuth 2.0 token-based authorization and has many benefits and improvements that help mitigate the issues in basic authentication. For example, OAuth access tokens have a limited usable lifetime, and are specific to the applications and resources for which they're issued, so they can't be reused. Entra ID also lets you deploy from other Azure services using managed identities.
+
+## Disable basic authentication
+
+### [Azure portal](#tab/portal)
+
+1. In the [Azure portal], search for and select **App Services**, and then select your app. 
+
+1. In the app's left menu, select **Configuration**.
+
+1. For **Basic Auth Publishing Credentials**, select **Off**, then select **Save**.
+
+    :::image type="content" source="media/configure-basic-auth-disable/basic-auth-disable.png" alt-text="A screenshot showing how to disable basic authentication for Azure App Service in the Azure portal.":::
+
+### [Azure CLI](#tab/cli)
+
+There are two different settings to configure when you disable basic authentication with Azure CLI, one for FTP and one for WebDeploy and Git.
+
+#### Disable for FTP
+
+To disable FTP access using basic authentication, you must have owner-level access to the app. Run the following CLI command by replacing the placeholders with your resource group name and app name:
+
+```azurecli-interactive
+az resource update --resource-group <group-name> --name ftp --namespace Microsoft.Web --resource-type basicPublishingCredentialsPolicies --parent sites/<app-name> --set properties.allow=false
+```
+
+#### Disable for WebDeploy and Git
+
+To disable basic authentication access to the WebDeploy port and the Git deploy URL (https://\<app-name>.scm.azurewebsites.net), run the following CLI command. Replace the placeholders with your resource group name and app name.
+
+```azurecli-interactive
+az resource update --resource-group <resource-group> --name scm --namespace Microsoft.Web --resource-type basicPublishingCredentialsPolicies --parent sites/<app-name> --set properties.allow=false
+```
+
+-----
+
+To confirm that FTP access is blocked, try [connecting to your app using FTP/S](deploy-ftp.md). You should get a `401 Unauthenticted` message.
+
+To confirm that Git access is blocked, try [local Git deployment](deploy-local-git.md). You should get an `Authentication failed` message.
+
+## Deployment without basic authentication
+
+When you disable basic authentication, deployment methods based on basic authentication stop working, such as FTP and local Git deployment. For alternate deployment methods, see [Authentication types by deployment methods in Azure App Service](deploy-authentication-types.md).
+
+<!-- Azure Pipelines with App Service deploy task (manual config) need the newer version hosted agent that supports vs2022.
+OIDC GitHub actions -->
+
+## Create a custom role with no permissions for basic authentication
+
+To prevent a lower-priveldged user from enabling basic authentication for any app, you can create a custom role and assign the user to the role.
+
+### [Azure portal](#tab/portal)
+
+1. In the Azure portal, in the top menu, search for and select the subscription you want to create the custom role in.
+1. From the left navigation, select **Access Control (IAM)** > **Add** > **Add custom role**.
+1. Set the **Basic** tab as you wish, then select **Next**.
+1. In the **Permissions** tab, and select **Exclude permissions**.
+1. Find and select **Microsoft Web Apps**, then search for the following operations:
+    
+    |Operation  |Description  |
+    |---------|---------|
+    |`microsoft.web/sites/basicPublishingCredentialsPolicies/ftp`     | FTP publishing credentials for App Service apps. |
+    |`microsoft.web/sites/basicPublishingCredentialsPolicies/scm`     | SCM publishing credentials for App Service apps. |
+    |`microsoft.web/sites/slots/basicPublishingCredentialsPolicies/ftp` | FTP publishing credentials for App Service slots. |
+    |`microsoft.web/sites/slots/basicPublishingCredentialsPolicies/scm` | SCM publishing credentials for App Service slots. |
+    
+1. Under each of these operations, select the box for **Write**, then select **Add**. This step adds the operation as **NotActions** for the role.
+
+    Your Permissions tab should look like the following screenshot:
+
+    :::image type="content" source="media/configure-basic-auth-disable/custom-role-no-basic-auth.png" alt-text="A screenshot showing the creation of a custom role with all basic authentication permissions excluded.":::
+
+1. Select **Review + create**, then select **Create**.
+
+1. You can now assign this role to your organization’s users.
+
+For more information, see [Create or update Azure custom roles using the Azure portal](../role-based-access-control/custom-roles-portal.md#step-2-choose-how-to-start)
+
+### [Azure CLI](#tab/cli)
+
+In the following command, replace *\<role-name>* and *\<subscription-guid>* (with the GUID of your subscription) and run in the cloud shell:
+
+```azurecli-interactive
+az role definition create --role-definition '{
+    "Name": "<role-name>",
+    "IsCustom": true,
+    "Description": "Prevents users from enabling basic authentication for all App Service apps or slots.",
+    "NotActions": [
+        "Microsoft.Web/sites/basicPublishingCredentialsPolicies/ftp/Write",
+        "Microsoft.Web/sites/basicPublishingCredentialsPolicies/scm/Write",
+        "Microsoft.Web/sites/slots/basicPublishingCredentialsPolicies/ftp/Write",
+        "Microsoft.Web/sites/slots/basicPublishingCredentialsPolicies/scm/Write"
+    ],
+    "AssignableScopes": ["/subscriptions/<subscription-guid>"]
+}'
+```
+
+You can now assign this role to your organization’s users.
+
+For more information, see [Create or update Azure custom roles using Azure CLI](../role-based-access-control/custom-roles-cli.md).
+
+-----
+
+## Monitor for basic authentication attempts
+
+All successful and attempted logins are logged to the Azure Monitor `AppServiceAuditLogs` log type. To audit the attempted and successful logins on FTP and WebDeploy, follow the steps at [Send logs to Azure Monitor](troubleshoot-diagnostic-logs.md#send-logs-to-azure-monitor) and enable shipping of the `AppServiceAuditLogs` log type.
+
+To confirm that the logs are shipped to your selected service(s), try logging in via FTP or WebDeploy. The following example shows a Storage Account log.
+
+<pre>
+{
+  "time": "2020-07-16T17:42:32.9322528Z",
+  "ResourceId": "/SUBSCRIPTIONS/EF90E930-9D7F-4A60-8A99-748E0EEA69DE/RESOURCEGROUPS/FREEBERGDEMO/PROVIDERS/MICROSOFT.WEB/SITES/FREEBERG-WINDOWS",
+  "Category": "AppServiceAuditLogs",
+  "OperationName": "Authorization",
+  "Properties": {
+    "User": "$freeberg-windows",
+    "UserDisplayName": "$freeberg-windows",
+    "UserAddress": "24.19.191.170",
+    "Protocol": "FTP"
+  }
+}
+</pre>
+
+## Basic authentication related policies
+
+[Azure Policy](../governance/policy/overview.md) can help you enforce organizational standards and to assess compliance at-scale. You can use Azure Policy to audit for any apps that still use basic authentication, and remediate any noncompliant resources. The following are built-in policies for auditing and remediating basic authentication on App Service:
+
+- [Audit policy for FTP](https://portal.azure.com/#blade/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2F871b205b-57cf-4e1e-a234-492616998bf7)
+- [Audit policy for SCM](https://ms.portal.azure.com/#blade/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2Faede300b-d67f-480a-ae26-4b3dfb1a1fdc)
+- [Remediation policy for FTP](https://ms.portal.azure.com/#blade/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2Ff493116f-3b7f-4ab3-bf80-0c2af35e46c2)
+- [Remediation policy for SCM](https://ms.portal.azure.com/#blade/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2F2c034a29-2a5f-4857-b120-f800fe5549ae)
+
+The following are corresponding policies for slots:
+
+- [Audit policy for FTP](https://ms.portal.azure.com/#blade/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2Fec71c0bc-6a45-4b1f-9587-80dc83e6898c)
+- [Audit policy for SCM](https://ms.portal.azure.com/#blade/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2F847ef871-e2fe-4e6e-907e-4adbf71de5cf)
+- [Remediation policy for FTP](https://ms.portal.azure.com/#blade/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2Ff493116f-3b7f-4ab3-bf80-0c2af35e46c2)
+- [Remediation policy for SCM](https://ms.portal.azure.com/#blade/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2F2c034a29-2a5f-4857-b120-f800fe5549ae)
+

articles/app-service/deploy-authentication-types.md

@@ -12,7 +12,7 @@ ms.author: cephalin
 Azure App Service lets you deploy your web application code and configuration by using multiple options. These deployment options may support one or more authentication mechanisms. This article provides details about various authentication mechanisms supported by different deployment methods. 
 
 > [!NOTE]
-> To disable basic authentication for your App Service app, see [Configure deployment credentials](deploy-configure-credentials.md). 
+> To disable basic authentication for your App Service app, see [Disable basic authentication in App Service deployments](configure-basic-auth-disable.md). 
 
 |Deployment method|Authentication  |Reference Documents |
 |:----|:----|:----|

articles/app-service/deploy-configure-credentials.md

@@ -135,33 +135,7 @@ Invoke-AzResourceAction -ResourceGroupName <group-name> -ResourceType Microsoft.
 
 ## Disable basic authentication
 
-Some organizations need to meet security requirements and would rather disable access via FTP or WebDeploy. This way, the organization's members can only access its App Services through APIs that are controlled by Microsoft Entra ID.
-
-### FTP
-
-To disable FTP access to the site, run the following CLI command. Replace the placeholders with your resource group and site name. 
-
-```azurecli-interactive
-az resource update --resource-group <resource-group> --name ftp --namespace Microsoft.Web --resource-type basicPublishingCredentialsPolicies --parent sites/<site-name> --set properties.allow=false
-```
-
-To confirm that FTP access is blocked, you can try to authenticate using an FTP client such as FileZilla. To retrieve the publishing credentials, go to the overview blade of your site and click Download Publish Profile. Use the file’s FTP hostname, username, and password to authenticate, and you will get a 401 error response, indicating that you are not authorized.
-
-### WebDeploy and SCM
-
-To disable basic auth access to the WebDeploy port and SCM site, run the following CLI command. Replace the placeholders with your resource group and site name. 
-
-```azurecli-interactive
-az resource update --resource-group <resource-group> --name scm --namespace Microsoft.Web --resource-type basicPublishingCredentialsPolicies --parent sites/<site-name> --set properties.allow=false
-```
-
-To confirm that the publish profile credentials are blocked on WebDeploy, try [publishing a web app using Visual Studio 2019](/visualstudio/deployment/quickstart-deploy-to-azure).
-
-### Disable access to the API
-
-The API in the previous section is backed Azure role-based access control (Azure RBAC), which means you can [create a custom role](../role-based-access-control/custom-roles.md#steps-to-create-a-custom-role) and assign lower-priveldged users to the role so they cannot enable basic auth on any sites. To configure the custom role, [follow these instructions](https://azure.github.io/AppService/2020/08/10/securing-data-plane-access.html#create-a-custom-rbac-role).
-
-You can also use [Azure Monitor](https://azure.github.io/AppService/2020/08/10/securing-data-plane-access.html#audit-with-azure-monitor) to audit any successful authentication requests and use [Azure Policy](https://azure.github.io/AppService/2020/08/10/securing-data-plane-access.html#enforce-compliance-with-azure-policy) to enforce this configuration for all sites in your subscription.
+See [Disable basic authentication in App Service deployments](configure-basic-auth-disable.md).
 
 ## Next steps
 

articles/app-service/media/configure-basic-auth-disable/basic-auth-disable.png

@@ -289,6 +289,8 @@
       href: tutorial-secure-ntier-app.md
     - name: Isolate network traffic (tutorial)
       href: tutorial-networking-isolate-vnet.md
+    - name: Disable basic auth
+      href: configure-basic-auth-disable.md
     - name: Security
       items:
         - name: Security recommendations