Merge branch 'main' of https://github.com/MicrosoftDocs/azure-docs-pr into apimllm

Author: dlepow

articles/ai-services/document-intelligence/concept-read.md

@@ -122,7 +122,7 @@ The searchable PDF capability enables you to convert an analog PDF, such as scan
   > [!IMPORTANT]
   >
   > * Currently, the searchable PDF capability is only supported by Read OCR model `prebuilt-read`. When using this feature, please specify the `modelId` as `prebuilt-read`, as other model types will return error for this preview version.
-  > * Searchable PDF is included with the 2024-07-31-preview `prebuilt-read` model with no usage cost for general PDF consumption.
+  > * Searchable PDF is included with the 2024-07-31-preview `prebuilt-read` model with no additional cost for generating a searchable PDF output.
 
 ### Use searchable PDF
 
@@ -135,7 +135,7 @@ POST /documentModels/prebuilt-read:analyze?output=pdf
 202
 ```
 
-Once the `Analyze` operation is complete, make a `GET` request to retrieve the `Analyze` operation results.
+Poll for completion of the `Analyze` operation. Once the operation is complete, issue a `GET` request to retrieve the PDF format of the `Analyze` operation results .
 
 Upon successful completion, the PDF can be retrieved and downloaded as `application/pdf`. This operation allows direct downloading of the embedded text form of PDF instead of Base64-encoded JSON.
 

articles/ai-services/document-intelligence/faq.yml

@@ -7,7 +7,7 @@ metadata:
   ms.service: azure-ai-document-intelligence
   ms.custom: references_regions
   ms.topic: faq
-  ms.date: 07/09/2024
+  ms.date: 08/08/2024
   ms.author: lajanuar
 title: Frequently asked questions
 summary: |
@@ -52,22 +52,23 @@ sections:
         answer: |
           **Yes.**
 
-          You can use a document generative AI solution to chat with your documents, generate captivating content from those documents, and access Azure OpenAI Service models on your data.
+          Document Intelligence now includes [custom generative](concept-custom.md) a new type of extraction model that uses Generative AI and large language models (LLMs) to extract fields from documents. In the past you've had to use a RAG (retrieval augmented generation) pattern to extract fields. The new model provides high quality results with a single API call.
+          You can also use a document generative AI solution to chat with your documents (RAG), generate captivating content from those documents, and access Azure OpenAI Service models on your data.
 
-          - With Azure AI Document Intelligence and Azure OpenAI combined, you can build an enterprise application to seamlessly interact with your documents by using natural languages. You can easily find answers, gain valuable insights, and generate new and engaging content from existing documents.
+          - With Azure AI Document Intelligence and Azure OpenAI combined, you can build an enterprise application to seamlessly interact with your documents using natural language. You can easily find answers, gain valuable insights, and generate new and engaging content from existing documents.
 
-          - You can find more details in the [technical community blog](https://techcommunity.microsoft.com/t5/azure-ai-services-blog/document-generative-ai-the-power-of-azure-ai-document/ba-p/3875015).
+          - You can find more details on the [retrieval augmented generation pattern here](concept-retrieval-augmented-generation.md).
 
       - question: |
          Can Document Intelligence help with semantic chunking within documents for retrieval-augmented generation?
         answer: |
           **Yes.**
           
-          Document Intelligence can provide the building blocks to enable semantic chunking. Semantic chunking is a key step in retrieval-augmented generation (RAG) to ensure its efficient storage and retrieval.
+          Document Intelligence can provide the building blocks to enable semantic chunking. Semantic chunking is a key step in retrieval-augmented generation (RAG) to ensure context dense chunks and relevence improvement.
 
-          - Document Intelligence provides a layout model that segments documents into coherent units based on their semantic content.
+          - Document Intelligence provides a layout model that provides an visual decomposition of the document into lines, paragraphs, sections, headers and footers.
 
-          - You can then export the obtained information to Markdown format, so that you can customize your semantic segmentation strategy based on the available building blocks.
+          - You can then choose to retrieve the results in markdown format, to further chunk the document on section or paragraph boundaries.
 
           For more information, see [overview of RAG in Document Intelligence](concept-retrieval-augmented-generation.md)
 
@@ -288,11 +289,17 @@ sections:
         The copy operation is limited to copying models within the specific cloud environment where you trained the model. For instance, copying models from the public cloud to the Azure Government cloud isn't supported.
 
     - question: |
-        Am I charged when training a custom neural model?
+        Am I charged when training a custom models?
       answer: |
-        **Yes for the first 10 hours only.**
+        **Yes**
 
-        If training hours exceed 10 hours, charges are incurred for further custom neural model training.
+        Although training is free for all custom generative and custom template models, creating the training dataset for all models requires running the Layout model on the training documents. Customers are responsible for this cost. 
+        
+        Custom generative models also rely on the auto label feature to speed up the generation of the labeled dataset. There is a cost associated with this action. While the build operation for template and generative models is free, creating the labeled dataset can result in some minimal costs.
+
+        Custom neural models have a limit on the number of models/the amount of time that models can be trained for free. The first 10 hours of training are free. If training a single model for longer than 10 hours or training multiple models that exceed the 10 hour limit, you will need to enable paid training by setting a training budget. See [training a custom neural model](concept-custom-neural.md) for details. 
+        
+        For v3.0 or v3.1 models the paid training tier only applies to additional models, the training time per model is not configurable.
 
   - name: Storage account
     questions:

articles/ai-services/document-intelligence/whats-new.md

@@ -37,12 +37,10 @@ The Document Intelligence [**2024-07-31-preview**](/rest/api/aiservices/document
 * **West Europe**
 * **North Central US**
 
-* [Read model](concept-read.md) now supports [PDF output](concept-read.md#searchable-pdf)  to download PDFs with embedded text from extraction results, allowing for PDF to be utilized in scenarios such as search and large language model ingestion.
-* [Layout model](concept-layout.md) now supports improved [figure detection](concept-layout.md#figures) where figures from documents can now be downloaded as an image file to be used for further figure understanding.
-* [Custom extraction models](concept-custom.md#custom-extraction-models)
-  * Custom extraction models now support updating the model in-place. 
-* [🆕 Custom generative (Document field extraction) model](concept-custom-generative.md)
-  * Document Intelligence now offers new custom generative model that utilizes generative AI to extract fields from unstructured documents or structured forms with a wide variety of visual templates.
+* [🆕 Document field extraction (custom generative) model](concept-custom-generative.md)
+  * Use **Generative AI** to extract fields from documents and forms. Document Intelligence now offers a new document field extraction model that utilizes large language models (LLMs) to extract fields from unstructured documents or structured forms with a wide variety of visual templates. With grounded values and confidence scores, the new Generative AI based extraction fits into your existing processes.
+* [🆕 Model compose with custom classifiers](concept-composed-models.md)
+  * Document Intelligence now adds support for composing model with an explicit custom classification model. [Learn more about the benefits](concept-composed-models.md) of using the new compose capability.
 * [Custom classification model](concept-custom.md#custom-classification-model)
   * Custom classification model now supports updating the model in-place as well.
   * Custom classification model adds support for model copy operation to enable backup and disaster recovery.
@@ -58,12 +56,14 @@ The Document Intelligence [**2024-07-31-preview**](/rest/api/aiservices/document
   * New prebuilt to extract account information including beginning and ending balances, transaction details from bank statements.​
 * [🆕 US Tax model](concept-tax-document.md)
   * New unified US tax model that can extract from forms such as W-2, 1098, 1099, and 1040.
+* 🆕 Searchable PDF. The [prebuilt read](concept-read.md) model now supports [PDF output](concept-read.md#searchable-pdf)  to download PDFs with embedded text from extraction results, allowing for PDF to be utilized in scenarios such as search copy of contents.
+* [Layout model](concept-layout.md) now supports improved [figure detection](concept-layout.md#figures) where figures from documents can now be downloaded as an image file to be used for further figure understanding. The layout model also features improvements to the OCR model for scanned text targeting improvements for single characters, boxed text and dense text documents.
+* [🆕 Batch API](concept-batch-analysis.md)
+  * Document Intelligence now adds support for batch analysis operation to support analyzing a set of documents to simplify developer experience and improve efficiency.
 * [Add-on capabilities](concept-add-on-capabilities.md)
   * [Query fields](concept-add-on-capabilities.md#query-fields) AI quality of extraction is improved with the latest model.
-* [🆕 Batch API](concept-batch-analysis.md)
-  * Document Intelligence now adds support for batch analysis operation to support analyzing a set of documents to simplify developer experience and improve service efficiency.
-* [🆕 Model compose with custom classifiers](concept-composed-models.md)
-  * Document Intelligence now adds support for composing model with an explicit custom classification model. 
+
+
 
 ## May 2024
 

articles/ai-services/openai/quotas-limits.md

@@ -10,7 +10,7 @@ ms.custom:
   - ignite-2023
   - references_regions
 ms.topic: conceptual
-ms.date: 08/06/2024
+ms.date: 08/08/2024
 ms.author: mbullwin
 ---
 
@@ -91,13 +91,23 @@ Global Standard deployments use Azure's global infrastructure, dynamically routi
 
 The Usage Limit determines the level of usage above which customers might see larger variability in response latency. A customer’s usage is defined per model and is the total tokens consumed across all deployments in all subscriptions in all regions for a given tenant.
 
+> [!NOTE]
+> Usage tiers only apply to standard and global standard deployment types. Usage tiers do not apply to global batch deployments.
+
 #### GPT-4o global standard & standard
 
 |Model| Usage Tiers per month |
 |----|----|
-|`gpt-4o` |1.5 Billion tokens |
+|`gpt-4o` | 8 Billion tokens |
 |`gpt-4o-mini` | 45 Billion tokens |
 
+#### GPT-4 standard
+
+|Model| Usage Tiers per month|
+|---|---|
+| `gpt-4` + `gpt-4-32k`  (all versions) | 4 Billion |
+
+
 ## Other offer types
 
 If your Azure subscription is linked to certain [offer types](https://azure.microsoft.com/support/legal/offer-details/) your max quota values are lower than the values indicated in the above tables.

articles/ai-services/speech-service/includes/release-notes/release-notes-containers.md

@@ -2,10 +2,27 @@
 author: eric-urban
 ms.service: azure-ai-speech
 ms.topic: include
-ms.date: 2/16/2024
+ms.date: 8/7/2024
 ms.author: eur
 ---
 
+### 2024-August release
+
+Add support for the latest model versions:
+- Speech language identification 1.14.0
+    - Upgrade .Net 8.0
+    - Mitigate Vulnerabilities
+- Neural text to speech 3.3.0
+    - Upgrade .Net 8.0
+    - Mitigate Vulnerabilities
+- Speech to text 4.18.0    
+    - Upgrade .Net 8.0
+    - Mitigate Vulnerabilities
+    - Upgrade Recognition Engine
+    - Fix the issue where `PropertyId.Speech_SegmentationSilenceTimeoutMs` was being ignored.
+    - Update Deprecated Models
+    - Remove the `uk-UA` locale
+
 ### 2024-February release
 
 Add support for the latest model versions:

articles/ai-services/speech-service/speech-container-overview.md

@@ -6,7 +6,7 @@ author: eric-urban
 manager: nitinme
 ms.service: azure-ai-speech
 ms.topic: how-to
-ms.date: 1/22/2024
+ms.date: 8/7/2024
 ms.author: eur
 keywords: on-premises, Docker, container
 ---
@@ -21,9 +21,9 @@ The following table lists the Speech containers available in the Microsoft Conta
 
 | Container | Features | Supported versions and locales |
 |--|--|--|
-| [Speech to text](speech-container-stt.md) | Transcribes continuous real-time speech or batch audio recordings with intermediate results.  | Latest: 4.7.0<br/><br/>For all supported versions and locales, see the [Microsoft Container Registry (MCR)](https://mcr.microsoft.com/product/azure-cognitive-services/speechservices/speech-to-text/tags) and [JSON tags](https://mcr.microsoft.com/v2/azure-cognitive-services/speechservices/speech-to-text/tags/list).|
-| [Custom speech to text](speech-container-cstt.md) | Using a custom model from the [custom speech portal](https://speech.microsoft.com/customspeech), transcribes continuous real-time speech or batch audio recordings into text with intermediate results. | Latest: 4.7.0<br/><br/>For all supported versions and locales, see the [Microsoft Container Registry (MCR)](https://mcr.microsoft.com/product/azure-cognitive-services/speechservices/custom-speech-to-text/tags) and [JSON tags](https://mcr.microsoft.com/v2/azure-cognitive-services/speechservices/speech-to-text/tags/list). |
-| [Speech language identification](speech-container-lid.md)<sup>1, 2</sup> | Detects the language spoken in audio files. | Latest: 1.13.0<br/><br/>For all supported versions and locales, see the [Microsoft Container Registry (MCR)](https://mcr.microsoft.com/product/azure-cognitive-services/speechservices/language-detection/tags) and [JSON tags](https://mcr.microsoft.com/v2/azure-cognitive-services/speechservices/language-detection/tags/list). |
+| [Speech to text](speech-container-stt.md) | Transcribes continuous real-time speech or batch audio recordings with intermediate results.  | Latest: 4.8.0<br/><br/>For all supported versions and locales, see the [Microsoft Container Registry (MCR)](https://mcr.microsoft.com/product/azure-cognitive-services/speechservices/speech-to-text/tags) and [JSON tags](https://mcr.microsoft.com/v2/azure-cognitive-services/speechservices/speech-to-text/tags/list).|
+| [Custom speech to text](speech-container-cstt.md) | Using a custom model from the [custom speech portal](https://speech.microsoft.com/customspeech), transcribes continuous real-time speech or batch audio recordings into text with intermediate results. | Latest: 4.8.0<br/><br/>For all supported versions and locales, see the [Microsoft Container Registry (MCR)](https://mcr.microsoft.com/product/azure-cognitive-services/speechservices/custom-speech-to-text/tags) and [JSON tags](https://mcr.microsoft.com/v2/azure-cognitive-services/speechservices/speech-to-text/tags/list). |
+| [Speech language identification](speech-container-lid.md)<sup>1, 2</sup> | Detects the language spoken in audio files. | Latest: 1.14.0<br/><br/>For all supported versions and locales, see the [Microsoft Container Registry (MCR)](https://mcr.microsoft.com/product/azure-cognitive-services/speechservices/language-detection/tags) and [JSON tags](https://mcr.microsoft.com/v2/azure-cognitive-services/speechservices/language-detection/tags/list). |
 | [Neural text to speech](speech-container-ntts.md) | Converts text to natural-sounding speech by using deep neural network technology, which allows for more natural synthesized speech. | Latest: 3.3.0<br/><br/>For all supported versions and locales, see the [Microsoft Container Registry (MCR)](https://mcr.microsoft.com/product/azure-cognitive-services/speechservices/neural-text-to-speech/tags) and [JSON tags](https://mcr.microsoft.com/v2/azure-cognitive-services/speechservices/neural-text-to-speech/tags/list). |
 
 <sup>1</sup> The container is available in public preview. Containers in preview are still under development and don't meet Microsoft's stability and support requirements.

articles/ai-studio/how-to/configure-managed-network.md

@@ -27,6 +27,7 @@ You need to configure following network isolation configurations.
 - Create private endpoint outbound rules to your private Azure resources. Private Azure AI Search isn't supported yet. 
 - If you use Visual Studio Code integration with allow only approved outbound mode, create FQDN outbound rules described in the [use Visual Studio Code](#scenario-use-visual-studio-code) section.
 - If you use HuggingFace models in Models with allow only approved outbound mode, create FQDN outbound rules described in the [use HuggingFace models](#scenario-use-huggingface-models) section.
+- If you use one of the open-source models with allow only approved outbound mode, create FQDN outbound rules described in the [curated by Azure AI](#scenario-curated-by-azure-ai) section.
 
 ## Network isolation architecture and isolation modes
 
@@ -759,6 +760,18 @@ If you plan to use __HuggingFace models__ with the hub, add outbound _FQDN_ rule
 * cnd.auth0.com
 * cdn-lfs.huggingface.co
 
+### Scenario: Curated by Azure AI
+
+These models involve dynamic installation of dependencies at runtime, and reequire outbound _FQDN_ rules to allow traffic to the following hosts:
+
+*.anaconda.org
+*.anaconda.com
+anaconda.com
+pypi.org
+*.pythonhosted.org
+*.pytorch.org
+pytorch.org
+
 ## Private endpoints
 
 Private endpoints are currently supported for the following Azure services:

articles/ai-studio/how-to/deploy-models-cohere-rerank.md

@@ -92,7 +92,7 @@ To create a deployment:
 1. Alternatively, you can initiate a deployment by starting from your project in AI Studio. 
 
     1. From the left sidebar of your project, select **Components** > **Deployments**.
-    1. Select **+ Create deployment**.
+    1. Select **+ Deploy model**.
     1. Search for and select **Cohere-rerank-3-english**. to open the Model Details page.
     1. Select **Confirm** to open a serverless API deployment window for the model.
 

articles/ai-studio/how-to/deploy-models-openai.md

@@ -48,7 +48,7 @@ Alternatively, you can initiate deployment by starting from your project in AI S
 
 1. Go to your project in AI Studio.
 1. Select **Components** > **Deployments**.
-1. Select **+ Create deployment**.
+1. Select **+ Deploy model**.
 1. In the **Collections** filter, select **Azure OpenAI**.
 1. Select a model such as `gpt-4` from the Azure OpenAI collection.
 1. Select **Confirm** to open the deployment window.

articles/ai-studio/how-to/deploy-models-serverless-availability.md

@@ -24,7 +24,10 @@ Certain models in the model catalog can be deployed as a serverless API with pay
 
 ## Region availability
 
-[!INCLUDE [region-availabilitity-serverless-api](~/reusable-content/ce-skilling/azure/includes/ai-studio/includes/region-availabilitity-serverless-api.md)]
+Pay-as-you-go billing is available only to users whose Azure subscription belongs to a billing account in a country where the model provider has made the offer available (see "offer availability region" in the table in the next section). If the offer is available in the relevant region, the user then must have a Hub/Project in the Azure region where the model is available for deployment or fine-tuning, as applicable (see "Hub/Project Region" columns in the following tables).
+
+[!INCLUDE [region-availability-maas](../includes/region-availability-maas.md)]
+
 
 ## Alternatives to region availability