Merge pull request #260767 from MicrosoftDocs/main

12/8/2023 PM Publish

Author: Taojunshen

articles/ai-services/language-service/personally-identifiable-information/includes/identification-entities.md

@@ -684,7 +684,7 @@ The following entities are grouped and listed by country/region:
 :::row:::
     :::column span="":::
 
-        EU GPU coordinates
+        EU GPS coordinates
 
     :::column-end:::
     :::column span="2":::

articles/ai-services/language-service/summarization/includes/regional-availability.md

@@ -13,8 +13,8 @@ ms.custom:
 ---
 
 > [!IMPORTANT]
-> One preview region, Sweden Central, showcases our latest and continually evolving LLM fine tuning techniques, where all summarization capabilities are available. We welcome customers to join our early adopter community by filling out the [access request form](https://aka.ms/applyforgatedsummarizationfeatures).
-. Your valuable feedback is vital to our continuous enhancement.
+> One preview region, Sweden Central, showcases our latest and continually evolving LLM fine tuning techniques, where all summarization capabilities are available. We welcome customers to join our early adopter community by filling out the [access request form](https://aka.ms/applyforgatedsummarizationfeatures). Your valuable feedback is vital to our continuous enhancement.
+>
 > Conversation summarization is only available using:
 >  - REST API
 >  - Python

articles/ai-services/language-service/summarization/language-support.md

@@ -48,9 +48,11 @@ Conversation summarization supports the following languages:
 | English               | `en`          |                     |
 | French                | `fr`          |                     |
 | German                | `de`          |                     |
+| Hebrew                | `he`          |                     |
 | Italian               | `it`          |                     |
 | Japanese              | `ja`          |                     |
 | Korean                | `ko`          |                     |
+| Polish                | `pl`          |                     |
 | Portuguese            | `pt`          |                     |
 | Spanish               | `es`          |                     |
 

articles/ai-services/openai/concepts/use-your-data.md

@@ -37,7 +37,7 @@ Azure OpenAI on your data supports the following filetypes:
 
 * `.txt`
 * `.md`
-* `.html` 
+* `.html`
 * Microsoft Word files
 * Microsoft PowerPoint files
 * PDF
@@ -134,8 +134,7 @@ Azure OpenAI on your data provides several search options you can use when you a
 
 > [!IMPORTANT]
 > * [Semantic search](/azure/search/semantic-search-overview#availability-and-pricing) and [vector search](https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/) are subject to additional pricing. You need to choose **Basic or higher SKU** to enable semantic search or vector search. See [pricing tier difference](/azure/search/search-sku-tier) and [service limits](/azure/search/search-limits-quotas-capacity) for more information.
-> * To help improve the quality of the information retrieval and model response, we recommend enabling [semantic search](/azure/search/semantic-search-overview) for the following languages: English, French, Spanish, Portuguese, Italian, Germany, Chinese(Zh), Japanese, Korean, Russian, Arabic
-> * If you enable vector search, you need to enable public network access for your Azure OpenAI resources.
+> * To help improve the quality of the information retrieval and model response, we recommend enabling [semantic search](/azure/search/semantic-search-overview) for the following languages: English, French, Spanish, Portuguese, Italian, Germany, Chinese(Zh), Japanese, Korean, Russian, and Arabic.
 
 | Search option       | Retrieval type | Additional pricing? |Benefits|
 |---------------------|------------------------|---------------------| -------- |
@@ -209,6 +208,27 @@ To add Azure Cosmos DB for MongoDB vCore as a data source, you will need an exis
 After ingesting your data, you can start chatting with the model on your data using the chat playground in Azure OpenAI studio, or the following methods:
 * [Web app](#using-the-web-app)
 * [REST API](../reference.md#azure-cosmos-db-for-mongodb-vcore)
+
+# [URL/web address](#tab/url-web)
+
+Currently, you can add your data from a URL/web address. Your data from a URL/web address needs to have the following characteristics to be properly ingested:
+
+* A public website, such as [Using your data with Azure OpenAI Service - Azure OpenAI | Microsoft Learn](/azure/ai-services/openai/concepts/use-your-data?tabs=ai-search). Note that you cannot add a URL/Web address with access control, such as with password.
+
+* A HTTPS website.
+
+* The size of content in each URL is smaller than 5MB.  
+
+* The website can be downloaded as one of the [supported file types](#data-formats-and-file-types).
+
+You can use URL as a data source in both the Azure OpenAI Studio and the [ingestion API](../reference.md#start-an-ingestion-job). To use URL/web address as a data source, you need to have an Azure AI Search resource and an Azure Blob Storage resource. When using the Ingestion API you need to create a container first. Only one layer of nested links is supported. Only up to 20 links, on the web page will be fetched.
+
+:::image type="content" source="../media/use-your-data/url.png" alt-text="A screenshot of the Azure OpenAI use your data url/webpage studio configuration page." lightbox="../media/use-your-data/url.png":::
+
+Once you have added the URL/web address for data ingestion, the web pages from your URL are fetched and saved to your Azure Blob Storage account with a container name: `webpage-<index name>`. Each URL will be saved into a different container within the account. Then the files are indexed into an Azure AI Search index, which is used for retrieval when you’re chatting with the model.
+
+When you want to reuse the same URL/web address, you can select [Azure AI Search](/azure/ai-services/openai/concepts/use-your-data?tabs=ai-search) as your data source and select the index you created with your URL previously. Then you can use the already indexed files instead of having the system crawl your URL again. You can use the Azure AI Search index directly and delete the storage container to free up your storage space.
+
 ---
 
 

articles/ai-services/openai/includes/connect-your-data-studio.md

@@ -42,11 +42,9 @@ Navigate to [Azure OpenAI Studio](https://oai.azure.com/) and sign-in with crede
 
 1. On the **Data management** pane, you can choose whether to enable [semantic search or vector search](../concepts/use-your-data.md#search-options) for your index.
 
-    > [!IMPORTANT]
-    > * Semantic search and vector search are subject to [additional pricing](../concepts/use-your-data.md#search-options).
-    >    * You can use *keyword* search as the search type for no additional cost.
-    > * To enable vector search, you will need a `text-embedding-ada-002` deployment in your Azure OpenAI resource.
-    > * Currently Azure OpenAI on your data supports semantic search for English data only. Only enable semantic search if both your documents and use case are in English.
+> [!IMPORTANT]
+> * [Semantic search](/azure/search/semantic-search-overview#availability-and-pricing) and [vector search](https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/) are subject to additional pricing. You need to choose **Basic or higher SKU** to enable semantic search or vector search. See [pricing tier difference](/azure/search/search-sku-tier) and [service limits](/azure/search/search-limits-quotas-capacity) for more information.
+> * To help improve the quality of the information retrieval and model response, we recommend enabling [semantic search](/azure/search/semantic-search-overview) for the following languages: English, French, Spanish, Portuguese, Italian, Germany, Chinese(Zh), Japanese, Korean, Russian, and Arabic.
     
 1. Review the details you entered, and select **Save and close**. You can now chat with the model and it will use information from your data to construct the response.
 

articles/ai-services/openai/media/use-your-data/url.png

@@ -530,16 +530,25 @@ curl -i -X PUT https://YOUR_RESOURCE_NAME.openai.azure.com/openai/extensions/on-
 } 
 ```
 
+**Header Parameters**
+
 |  Parameters | Type | Required? | Default | Description |
 |---|---|---|---|---|
-| `dataRefreshIntervalInMinutes` | string | Required | 0 | The data refresh interval in minutes. If you want to run a single ingestion job without a schedule, set this parameter to `0`. |
-| `completionAction` | string | Optional | `cleanUpAssets` | What should happen to the assets created during the ingestion process upon job completion. Valid values are `cleanUpAssets` or `keepAllAssets`. `keepAllAssets` leaves all the intermediate assets for users interested in reviewing the intermediate results, which can be helpful for debugging assets. `cleanUpAssets` removes the assets after job completion. |
-| `searchServiceEndpoint` | string | Required |null | The endpoint of the search resource in which the data will be ingested. |
+| `searchServiceEndpoint` | string | Required |null | The endpoint of the search resource in which the data will be ingested.|
 | `searchServiceAdminKey` | string | Optional | null | If provided, the key will be used to authenticate with the `searchServiceEndpoint`. If not provided, the system-assigned identity of the Azure OpenAI resource will be used. In this case, the system-assigned identity must have "Search Service Contributor" role assignment on the search resource. |
 | `storageConnectionString` | string | Required | null | The connection string for the storage account where the input data is located. An account key has to be provided in the connection string. It should look something like `DefaultEndpointsProtocol=https;AccountName=<your storage account>;AccountKey=<your account key>` |
 | `storageContainer` | string | Required | null | The name of the container where the input data is located. | 
 | `embeddingEndpoint` | string | Optional | null | Not required if you use semantic or only keyword search. It is required if you use vector, hybrid, or hybrid + semantic search |
 | `embeddingKey` | string | Optional | null | The key of the embedding endpoint. This is required if the embedding endpoint is not empty. |
+| `url` | string | Optional | null | If URL is not null, the provided url will be crawled into the provided storage container and then ingested accordingly.|
+
+**Body Parameters**
+
+|  Parameters | Type | Required? | Default | Description |
+|---|---|---|---|---|
+| `dataRefreshIntervalInMinutes` | string | Required | 0 | The data refresh interval in minutes. If you want to run a single ingestion job without a schedule, set this parameter to `0`. |
+| `completionAction` | string | Optional | `cleanUpAssets` | What should happen to the assets created during the ingestion process upon job completion. Valid values are `cleanUpAssets` or `keepAllAssets`. `keepAllAssets` leaves all the intermediate assets for users interested in reviewing the intermediate results, which can be helpful for debugging assets. `cleanUpAssets` removes the assets after job completion. |
+| `chunkSize` | int | Optional |1024 |This number defines the maximum number of tokens in each chunk produced by the ingestion flow. |
 
 
 ### List ingestion jobs

articles/ai-services/openai/reference.md

@@ -18,7 +18,7 @@ ms.author: eur
 
 The Azure AI command-line interface (CLI) is a cross-platform command-line tool to connect to Azure AI services and execute control-plane and data-plane operations without having to write any code. The Azure AI CLI allows the execution of commands through a terminal using interactive command-line prompts or via script. 
 
-You can easily use the Azure AI CLI to experiment with key Azure AI service features and see how they work with your use cases. Within minutes, you can set up all the required Azure resources needed, and build a customized Copilot using OpenAI's chat completions APIs and your own data. You can try it out interactively, or script larger processes to automate your own workflows and evaluations as part of your CI/CD system.
+You can easily use the Azure AI CLI to experiment with key Azure AI features and see how they work with your use cases. Within minutes, you can set up all the required Azure resources needed, and build a customized copilot using Azure OpenAI chat completions APIs and your own data. You can try it out interactively, or script larger processes to automate your own workflows and evaluations as part of your CI/CD system.
 
 ## Prerequisites
 
@@ -36,29 +36,52 @@ The following set of commands are provided for a few popular operating systems.
 
 # [Windows](#tab/windows)
 
-To install the .NET SDK, Azure CLI, and Azure AI CLI, run the following commands in a PowerShell terminal. Skip any that you don't need. 
+To install the .NET SDK, Azure CLI, and Azure AI CLI, run the following command. 
 
 ```bash
 dotnet tool install --prerelease --global Azure.AI.CLI
 ```
 
+To update the Azure AI CLI, run the following command:
+
+```bash
+dotnet tool update --prerelease --global Azure.AI.CLI
+```
+
 # [Linux](#tab/linux)
 
-On Debian and Ubuntu, run:
+To install the .NET SDK, Azure CLI, and Azure AI CLI on Debian and Ubuntu, run the following command:
 
 ```
 curl -sL https://aka.ms/InstallAzureAICLIDeb | bash
 ```
 
-# [macOS](#tab/macos)
+Alternatively, you can run the following command:
+
+```bash
+dotnet tool install --prerelease --global Azure.AI.CLI
+```
 
-On macOS, you can use *homebrew* and *wget*. For example, run the following commands in a terminal:
+To update the Azure AI CLI, run the following command:
 
+```bash
+dotnet tool update --prerelease --global Azure.AI.CLI
+```
+
+# [macOS](#tab/macos)
+
+To install the .NET SDK, Azure CLI, and Azure AI CLI on macOS 10.14 or later, run the following command:
 
 ```bash
 dotnet tool install --prerelease --global Azure.AI.CLI
 ```
 
+To update the Azure AI CLI, run the following command:
+
+```bash
+dotnet tool update --prerelease --global Azure.AI.CLI
+```
+
 ---
 
 ## Run the Azure AI CLI without installing it
@@ -109,8 +132,8 @@ The following table describes the scenarios for each flow.
 
 | Scenario | Description |
 | --- | --- |
-| Initialize a new AI project | Choose if you don't have an existing AI project that you have been working with in the Azure AI Studio. `ai init` walks you through creating or attaching resources. |
-| Initialize an existing AI project | Choose if you have an existing AI project you want to work with. `ai init` checks your existing linked resources, and ask you to set anything that hasn't been set before. |
+| Initialize a new AI project | Choose if you don't have an existing AI project that you have been working with in the Azure AI Studio. The `ai init` command walks you through creating or attaching resources. |
+| Initialize an existing AI project | Choose if you have an existing AI project you want to work with. The `ai init` command checks your existing linked resources, and ask you to set anything that hasn't been set before. |
 | Initialize standalone resources| Choose if you're building a simple solution connected to a single AI service, or if you want to attach more resources to your development environment |
 
 Working with an AI project is recommended when using the Azure AI Studio and/or connecting to multiple AI services. Projects come with an AI Resource that houses related projects and shareable resources like compute and connections to services. Projects also allow you to connect code to cloud resources (storage and model deployments), save evaluation results, and host code behind online endpoints. You're prompted to create and/or attach Azure AI Services to your project.
@@ -151,11 +174,11 @@ The following resources can be initialized standalone, or attached to projects:
 
 ## Project connections
 
-When working the Azure AI CLI, you'll want to use your project's connections. Connections are established to attached resources and allow you to integrate services with your project. You can have project-specific connections, or connections shared at the Azure AI resource level. For more information, see [Azure AI resources](../concepts/ai-resources.md) and [connections](../concepts/connections.md).
+When working the Azure AI CLI, you want to use your project's connections. Connections are established to attached resources and allow you to integrate services with your project. You can have project-specific connections, or connections shared at the Azure AI resource level. For more information, see [Azure AI resources](../concepts/ai-resources.md) and [connections](../concepts/connections.md).
 
 When you run `ai init` your project connections get set in your development environment, allowing seamless integration with AI services. You can view these connections by running `ai service connection list`, and further manage these connections with `ai service connection` subcommands.
 
-Any updates you make to connections in the AI CLI will be reflected in the AI Studio, and vice versa.
+Any updates you make to connections in the Azure AI CLI is reflected in [Azure AI Studio](https://ai.azure.com), and vice versa.
 
 ## ai dev