Merge branch 'main' of https://github.com/seesharprun/azure-docs-pr into cosmos-ttl-gary-hope

Author: seesharprun

articles/ai-services/document-intelligence/concept-read.md

@@ -5,8 +5,6 @@ description: Extract print and handwritten text from scanned and digital documen
 author: laujan
 manager: nitinme
 ms.service: azure-ai-document-intelligence
-ms.custom:
-  - ignite-2023
 ms.topic: conceptual
 ms.date: 08/07/2024
 ms.author: lajanuar
@@ -122,7 +120,7 @@ The searchable PDF capability enables you to convert an analog PDF, such as scan
   > [!IMPORTANT]
   >
   > * Currently, the searchable PDF capability is only supported by Read OCR model `prebuilt-read`. When using this feature, please specify the `modelId` as `prebuilt-read`, as other model types will return error for this preview version.
-  > * Searchable PDF is included with the 2024-07-31-preview `prebuilt-read` model with no usage cost for general PDF consumption.
+  > * Searchable PDF is included with the 2024-07-31-preview `prebuilt-read` model with no additional cost for generating a searchable PDF output.
 
 ### Use searchable PDF
 
@@ -135,7 +133,7 @@ POST /documentModels/prebuilt-read:analyze?output=pdf
 202
 ```
 
-Once the `Analyze` operation is complete, make a `GET` request to retrieve the `Analyze` operation results.
+Poll for completion of the `Analyze` operation. Once the operation is complete, issue a `GET` request to retrieve the PDF format of the `Analyze` operation results.
 
 Upon successful completion, the PDF can be retrieved and downloaded as `application/pdf`. This operation allows direct downloading of the embedded text form of PDF instead of Base64-encoded JSON.
 

articles/ai-services/document-intelligence/faq.yml

@@ -7,7 +7,7 @@ metadata:
   ms.service: azure-ai-document-intelligence
   ms.custom: references_regions
   ms.topic: faq
-  ms.date: 07/09/2024
+  ms.date: 08/08/2024
   ms.author: lajanuar
 title: Frequently asked questions
 summary: |
@@ -52,22 +52,23 @@ sections:
         answer: |
           **Yes.**
 
-          You can use a document generative AI solution to chat with your documents, generate captivating content from those documents, and access Azure OpenAI Service models on your data.
+          Document Intelligence now includes [custom generative](concept-custom.md) a new type of extraction model that uses Generative AI and large language models (LLMs) to extract fields from documents. In the past, you used a RAG (retrieval augmented generation) pattern to extract fields. The new model provides high quality results with a single API call.
+          You can also use a document generative AI solution to chat with your documents (RAG), generate captivating content from those documents, and access Azure OpenAI Service models on your data.
 
-          - With Azure AI Document Intelligence and Azure OpenAI combined, you can build an enterprise application to seamlessly interact with your documents by using natural languages. You can easily find answers, gain valuable insights, and generate new and engaging content from existing documents.
+          - With Azure AI Document Intelligence and Azure OpenAI combined, you can build an enterprise application to seamlessly interact with your documents using natural language. You can easily find answers, gain valuable insights, and generate new and engaging content from existing documents.
 
-          - You can find more details in the [technical community blog](https://techcommunity.microsoft.com/t5/azure-ai-services-blog/document-generative-ai-the-power-of-azure-ai-document/ba-p/3875015).
+          - You can find more details on the [retrieval augmented generation pattern here](concept-retrieval-augmented-generation.md).
 
       - question: |
          Can Document Intelligence help with semantic chunking within documents for retrieval-augmented generation?
         answer: |
           **Yes.**
           
-          Document Intelligence can provide the building blocks to enable semantic chunking. Semantic chunking is a key step in retrieval-augmented generation (RAG) to ensure its efficient storage and retrieval.
+          Document Intelligence can provide the building blocks to enable semantic chunking. Semantic chunking is a key step in retrieval-augmented generation (RAG) to ensure context dense chunks and relevance improvement.
 
-          - Document Intelligence provides a layout model that segments documents into coherent units based on their semantic content.
+          - Document Intelligence provides a layout model that provides a visual decomposition of the document into lines, paragraphs, sections, headers, and footers.
 
-          - You can then export the obtained information to Markdown format, so that you can customize your semantic segmentation strategy based on the available building blocks.
+          - You can then choose to retrieve the results in markdown format, to further chunk the document on section or paragraph boundaries.
 
           For more information, see [overview of RAG in Document Intelligence](concept-retrieval-augmented-generation.md)
 
@@ -288,11 +289,17 @@ sections:
         The copy operation is limited to copying models within the specific cloud environment where you trained the model. For instance, copying models from the public cloud to the Azure Government cloud isn't supported.
 
     - question: |
-        Am I charged when training a custom neural model?
+        Am I charged when training a custom models?
       answer: |
-        **Yes for the first 10 hours only.**
+        **Yes**
 
-        If training hours exceed 10 hours, charges are incurred for further custom neural model training.
+        Although training is free for all custom generative and custom template models, creating the training dataset for all models requires running the Layout model on the training documents. Customers are responsible for this cost. 
+        
+        Custom generative models also rely on the auto label feature to speed up the generation of the labeled dataset. There's a cost associated with this action. While the build operation for template and generative models is free, creating the labeled dataset can result in some minimal costs.
+
+        Custom neural models have a limit on the number of models/the amount of time that models can be trained for free. The first 10 hours of training are free. If training a single model for longer than 10 hours or training multiple models that exceed the 10 hour limit, you need to enable paid training by setting a training budget. See [training a custom neural model](concept-custom-neural.md) for details. 
+        
+        For v3.0 or v3.1 models the paid training tier only applies to added models, the training time per model isn't configurable.
 
   - name: Storage account
     questions:

articles/ai-services/document-intelligence/whats-new.md

@@ -37,12 +37,10 @@ The Document Intelligence [**2024-07-31-preview**](/rest/api/aiservices/document
 * **West Europe**
 * **North Central US**
 
-* [Read model](concept-read.md) now supports [PDF output](concept-read.md#searchable-pdf)  to download PDFs with embedded text from extraction results, allowing for PDF to be utilized in scenarios such as search and large language model ingestion.
-* [Layout model](concept-layout.md) now supports improved [figure detection](concept-layout.md#figures) where figures from documents can now be downloaded as an image file to be used for further figure understanding.
-* [Custom extraction models](concept-custom.md#custom-extraction-models)
-  * Custom extraction models now support updating the model in-place. 
-* [🆕 Custom generative (Document field extraction) model](concept-custom-generative.md)
-  * Document Intelligence now offers new custom generative model that utilizes generative AI to extract fields from unstructured documents or structured forms with a wide variety of visual templates.
+* [🆕 Document field extraction (custom generative) model](concept-custom-generative.md)
+  * Use **Generative AI** to extract fields from documents and forms. Document Intelligence now offers a new document field extraction model that utilizes large language models (LLMs) to extract fields from unstructured documents or structured forms with a wide variety of visual templates. With grounded values and confidence scores, the new Generative AI based extraction fits into your existing processes.
+* [🆕 Model compose with custom classifiers](concept-composed-models.md)
+  * Document Intelligence now adds support for composing model with an explicit custom classification model. [Learn more about the benefits](concept-composed-models.md) of using the new compose capability.
 * [Custom classification model](concept-custom.md#custom-classification-model)
   * Custom classification model now supports updating the model in-place as well.
   * Custom classification model adds support for model copy operation to enable backup and disaster recovery.
@@ -58,12 +56,14 @@ The Document Intelligence [**2024-07-31-preview**](/rest/api/aiservices/document
   * New prebuilt to extract account information including beginning and ending balances, transaction details from bank statements.​
 * [🆕 US Tax model](concept-tax-document.md)
   * New unified US tax model that can extract from forms such as W-2, 1098, 1099, and 1040.
+* 🆕 Searchable PDF. The [prebuilt read](concept-read.md) model now supports [PDF output](concept-read.md#searchable-pdf)  to download PDFs with embedded text from extraction results, allowing for PDF to be utilized in scenarios such as search copy of contents.
+* [Layout model](concept-layout.md) now supports improved [figure detection](concept-layout.md#figures) where figures from documents can now be downloaded as an image file to be used for further figure understanding. The layout model also features improvements to the OCR model for scanned text targeting improvements for single characters, boxed text, and dense text documents.
+* [🆕 Batch API](concept-batch-analysis.md)
+  * Document Intelligence now adds support for batch analysis operation to support analyzing a set of documents to simplify developer experience and improve efficiency.
 * [Add-on capabilities](concept-add-on-capabilities.md)
   * [Query fields](concept-add-on-capabilities.md#query-fields) AI quality of extraction is improved with the latest model.
-* [🆕 Batch API](concept-batch-analysis.md)
-  * Document Intelligence now adds support for batch analysis operation to support analyzing a set of documents to simplify developer experience and improve service efficiency.
-* [🆕 Model compose with custom classifiers](concept-composed-models.md)
-  * Document Intelligence now adds support for composing model with an explicit custom classification model. 
+
+
 
 ## May 2024
 

articles/ai-studio/how-to/configure-managed-network.md

@@ -27,6 +27,7 @@ You need to configure following network isolation configurations.
 - Create private endpoint outbound rules to your private Azure resources. Private Azure AI Search isn't supported yet. 
 - If you use Visual Studio Code integration with allow only approved outbound mode, create FQDN outbound rules described in the [use Visual Studio Code](#scenario-use-visual-studio-code) section.
 - If you use HuggingFace models in Models with allow only approved outbound mode, create FQDN outbound rules described in the [use HuggingFace models](#scenario-use-huggingface-models) section.
+- If you use one of the open-source models with allow only approved outbound mode, create FQDN outbound rules described in the [curated by Azure AI](#scenario-curated-by-azure-ai) section.
 
 ## Network isolation architecture and isolation modes
 
@@ -759,6 +760,18 @@ If you plan to use __HuggingFace models__ with the hub, add outbound _FQDN_ rule
 * cnd.auth0.com
 * cdn-lfs.huggingface.co
 
+### Scenario: Curated by Azure AI
+
+These models involve dynamic installation of dependencies at runtime, and reequire outbound _FQDN_ rules to allow traffic to the following hosts:
+
+*.anaconda.org
+*.anaconda.com
+anaconda.com
+pypi.org
+*.pythonhosted.org
+*.pytorch.org
+pytorch.org
+
 ## Private endpoints
 
 Private endpoints are currently supported for the following Azure services:

articles/ai-studio/how-to/deploy-models-cohere-rerank.md

@@ -92,7 +92,7 @@ To create a deployment:
 1. Alternatively, you can initiate a deployment by starting from your project in AI Studio. 
 
     1. From the left sidebar of your project, select **Components** > **Deployments**.
-    1. Select **+ Create deployment**.
+    1. Select **+ Deploy model**.
     1. Search for and select **Cohere-rerank-3-english**. to open the Model Details page.
     1. Select **Confirm** to open a serverless API deployment window for the model.
 

articles/ai-studio/how-to/deploy-models-openai.md

@@ -48,7 +48,7 @@ Alternatively, you can initiate deployment by starting from your project in AI S
 
 1. Go to your project in AI Studio.
 1. Select **Components** > **Deployments**.
-1. Select **+ Create deployment**.
+1. Select **+ Deploy model**.
 1. In the **Collections** filter, select **Azure OpenAI**.
 1. Select a model such as `gpt-4` from the Azure OpenAI collection.
 1. Select **Confirm** to open the deployment window.

articles/ai-studio/how-to/deploy-models-serverless-availability.md

@@ -24,7 +24,10 @@ Certain models in the model catalog can be deployed as a serverless API with pay
 
 ## Region availability
 
-[!INCLUDE [region-availabilitity-serverless-api](~/reusable-content/ce-skilling/azure/includes/ai-studio/includes/region-availabilitity-serverless-api.md)]
+Pay-as-you-go billing is available only to users whose Azure subscription belongs to a billing account in a country where the model provider has made the offer available (see "offer availability region" in the table in the next section). If the offer is available in the relevant region, the user then must have a Hub/Project in the Azure region where the model is available for deployment or fine-tuning, as applicable (see "Hub/Project Region" columns in the following tables).
+
+[!INCLUDE [region-availability-maas](../includes/region-availability-maas.md)]
+
 
 ## Alternatives to region availability
 

articles/ai-studio/how-to/deploy-models-timegen-1.md

@@ -92,7 +92,7 @@ These steps demonstrate the deployment of TimeGEN-1. To create a deployment:
 1. Select **Deploy** to open a serverless API deployment window for the model.
 1. Alternatively, you can initiate a deployment by starting from your project in AI Studio.
     1. From the left sidebar of your project, select **Components** > **Deployments**.
-    1. Select **+ Create deployment**.
+    1. Select **+ Deploy model**.
     1. Search for and select **TimeGEN-1**. to open the Model's Details page.
     1. Select **Confirm** to open a serverless API deployment window for the model.
 1. Select the project in which you want to deploy your model. To deploy the TimeGEN-1 model, your project must be in one of the regions listed in the [Prerequisites](#prerequisites) section.

articles/ai-studio/how-to/model-catalog-overview.md

@@ -143,26 +143,7 @@ In Azure AI Studio, you can use vector indexes and retrieval-augmented generatio
 
 ### Regional availability of offers and models
 
-Pay-as-you-go billing is available only if your Azure subscription belongs to a billing account in a country where the model provider has made the offer available. If the offer is available in the relevant region, you must have a hub or project in the Azure region where the model is available for deployment or fine-tuning, as applicable.
-
-<!-- docutune:disable -->
-
-Model | Offer availability region | Hub/Project region for deployment | Hub/project region for fine-tuning
---|--|--|--
-Llama-3-70B-Instruct <br> Llama-3-8B-Instruct | [Microsoft-managed countries/regions](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) | East US, East US 2,  North Central US, South Central US, Sweden Central, West US,  West US 3 | Not available
-Llama-2-7b <br> Llama-2-13b <br> Llama-2-70b | [Microsoft-managed countries/regions](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) | East US, East US 2, North Central US, South Central US, West US, West US 3 | West US 3
-Llama-2-7b-chat <br> Llama-2-13b-chat <br> Llama-2-70b-chat | [Microsoft-managed countries/regions](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) | East US, East US 2, North Central US, South Central US, West US, West US 3 | Not available
-Mistral Small | [Microsoft-managed countries/regions](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) | East US, East US 2, North Central US, South Central US, Sweden Central, West US, West US 3 | Not available
-Mistral Large (2402) <br> Mistral-Large (2407) | [Microsoft-managed countries/regions](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) <br> Brazil <br> Hong Kong <br> Israel| East US, East US 2, North Central US, South Central US, Sweden Central, West US, West US 3 | Not available
-Mistral NeMo | [Microsoft-managed countries/regions](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) <br> Brazil <br> Hong Kong <br> Israel| East US, East US 2, North Central US, South Central US, Sweden Central, West US, West US 3 | Not available
-Cohere-command-r-plus <br> Cohere-command-r <br> Cohere-embed-v3-english <br> Cohere-embed-v3-multilingual | [Microsoft-managed countries/regions](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) <br> Japan |  East US, East US 2, North Central US, South Central US, Sweden Central, West US, West US 3 | Not available
-TimeGEN-1 | [Microsoft-managed countries/regions](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) <br> Mexico <br> Israel| East US, East US 2, North Central US, South Central US, Sweden Central, West US, West US 3 | Not available
-jais-30b-chat | [Microsoft-managed countries/regions](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) | East US, East US 2, North Central US, South Central US, Sweden Central, West US, West US 3 | Not available
-Phi-3-mini-4k-instruct <br> Phi-3-mini-128k-instruct | [Microsoft-managed countries/regions](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) | East US 2, Sweden Central | Not available
-Phi-3-small-8k-instruct <br> Phi-3-small-128k-Instruct | [Microsoft-managed countries/regions](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) | East US 2, Sweden Central | Not available
-Phi-3-medium-4k-instruct <br> Phi-3-medium-128k-instruct | [Microsoft-managed countries/regions](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) | East US 2, Sweden Central | Not available
-
-<!-- docutune:enable -->
+Pay-as-you-go billing is available only to users whose Azure subscription belongs to a billing account in a country where the model provider has made the offer available. If the offer is available in the relevant region, the user then must have a Hub/Project in the Azure region where the model is available for deployment or fine-tuning, as applicable. See [Region availability for models in serverless API endpoints | Azure AI Studio](deploy-models-serverless-availability.md) for detailed information.
 
 ### Content safety for models deployed via serverless APIs