merge from main

Author: PatrickFarley

articles/ai-services/openai/how-to/gpt-with-vision.md

@@ -101,10 +101,7 @@ The API response should look like the following.
     ],
     "choices": [
         {
-            "finish_details": {
-                "type": "stop",
-                "stop": "<|fim_suffix|>"
-            },
+            "finish_reason":"stop",
             "index": 0,
             "message": {
                 "role": "assistant",
@@ -138,13 +135,11 @@ The API response should look like the following.
 }
 ```
 
-Every response includes a `"finish_details"` field. The subfield `"type"` has the following possible values:
+Every response includes a `"finish_reason"` field. It has the following possible values:
 - `stop`: API returned complete model output.
-- `max_tokens`: Incomplete model output due to the `max_tokens` input parameter or model's token limit.
+- `length`: Incomplete model output due to the `max_tokens` input parameter or model's token limit.
 - `content_filter`: Omitted content due to a flag from our content filters.
 
-If `finish_details.type` is `stop`, then there is another `"stop"` property that specifies the token that caused the output to end.
-
 ## Detail parameter settings in image processing: Low, High, Auto  
 
 The detail parameter in the model offers three choices: `low`, `high`, or `auto`, to adjust the way the model interprets and processes images. The default setting is auto, where the model decides between low or high based on the size of the image input. 
@@ -239,11 +234,7 @@ The chat responses you receive from the model should now include enhanced inform
     "choices":
     [
         {
-            "finish_details":
-            {
-                "type": "stop",
-                "stop": "<|fim_suffix|>"
-            },
+            "finish_reason":"stop",
             "index": 0,
             "message":
             {
@@ -283,13 +274,11 @@ The chat responses you receive from the model should now include enhanced inform
 }
 ```
 
-Every response includes a `"finish_details"` field. The subfield `"type"` has the following possible values:
+Every response includes a `"finish_reason"` field. It has the following possible values:
 - `stop`: API returned complete model output.
-- `max_tokens`: Incomplete model output due to the `max_tokens` input parameter or model's token limit.
+- `length`: Incomplete model output due to the `max_tokens` input parameter or model's token limit.
 - `content_filter`: Omitted content due to a flag from our content filters.
 
-If `finish_details.type` is `stop`, then there is another `"stop"` property that specifies the token that caused the output to end.
-
 ## Use Vision enhancement with video
 
 GPT-4 Turbo with Vision provides exclusive access to Azure AI Services tailored enhancements. The **video prompt** integration uses Azure AI Vision video retrieval to sample a set of frames from a video and create a transcript of the speech in the video. It enables the AI model to give summaries and answers about video content.
@@ -377,11 +366,7 @@ The chat responses you receive from the model should include information about t
     "choices":
     [
         {
-            "finish_details":
-            {
-                "type": "stop",
-                "stop": "<|fim_suffix|>"
-            },
+            "finish_reason":"stop",
             "index": 0,
             "message":
             {
@@ -399,13 +384,56 @@ The chat responses you receive from the model should include information about t
 }
 ```
 
-Every response includes a `"finish_details"` field. The subfield `"type"` has the following possible values:
+Every response includes a `"finish_reason"` field. It has the following possible values:
 - `stop`: API returned complete model output.
-- `max_tokens`: Incomplete model output due to the `max_tokens` input parameter or model's token limit.
+- `length`: Incomplete model output due to the `max_tokens` input parameter or model's token limit.
 - `content_filter`: Omitted content due to a flag from our content filters.
 
-If `finish_details.type` is `stop`, then there is another `"stop"` property that specifies the token that caused the output to end.
+### Pricing example for Video prompts
+The pricing for GPT-4 Turbo with Vision is dynamic and depends on the specific features and inputs used. For a comprehensive view of Azure OpenAI pricing see [Azure OpenAI Pricing](https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/).
+
+The base charges and additional features are outlined below:
+
+Base Pricing for GPT-4 Turbo with Vision is:
+- Input: $0.01 per 1000 tokens
+- Output: $0.03 per 1000 tokens
+  
+Video prompt integration with Video Retrieval Add-on:
+- Ingestion: $0.05 per minute of video
+- Transactions: $0.25 per 1000 queries of the Video Retrieval index
+
+Processing videos will involve the use of extra tokens to identify key frames for analysis. The number of these additional tokens will be roughly equivalent to the sum of the tokens in the text input plus 700 tokens.
+
+#### Calculation
+For a typical use case let's imagine that I have use a 3-minute video with a 100-token prompt input. The section of video has a transcript that's 100-tokens long and when I process the prompt, I generate 100-tokens of output. The pricing for this transaction would be as follows:
+
+| Item                                      | Detail                                                        | Total Cost   |
+|-------------------------------------------|---------------------------------------------------------------|--------------|
+| GPT-4 Turbo with Vision Input Tokens      | 100 text tokens                                               | $0.001       |
+| Additional Cost to identify frames        | 100 input tokens + 700 tokens + 1 Video Retrieval txn         | $0.00825     |
+| Image Inputs and Transcript Input         | 20 images (85 tokens each) + 100 transcript tokens            | $0.018       |
+| Output Tokens                             | 100 tokens (assumed)                                          | $0.003       |
+| **Total Cost**                            |                                                               | **$0.03025** |
+
+Additionally, there's a one-time indexing cost of $0.15 to generate the Video Retrieval index for this 3-minute segment of video. This index can be reused across any number of Video Retrieval and GPT-4 Turbo with Vision calls.
+
+## Limitations
+
+### Image support
+
+- **Limitation on image enhancements per chat session**: Enhancements cannot be applied to multiple images within a single chat call.
+- **Maximum input image size**: The maximum size for input images is restricted to 20 MB.
+- **Object grounding in enhancement API**: When the enhancement API is used for object grounding, and the model detects duplicates of an object, it will generate one bounding box and label for all the duplicates instead of separate ones for each.
+- **Low resolution accuracy**: When images are analyzed using the "low resolution" setting, it allows for faster responses and uses fewer input tokens for certain use cases. However, this could impact the accuracy of object and text recognition within the image.
+- **Image chat restriction**: When uploading images in the chat playground or the API, there is a limit of 10 images per chat call.
+
+### Video support
 
+- **Low resolution**: Video frames are analyzed using GPT-4 Turbo with Vision's "low resolution" setting, which may affect the accuracy of small object and text recognition in the video.
+- **Video file limits**: Both MP4 and MOV file types are supported. In the Azure AI Playground, videos must be less than 3 minutes long. When you use the API there is no such limitation.
+- **Prompt limits**: Video prompts only contain one video and no images. In Playground, you can clear the session to try another video or images.
+- **Limited frame selection**: The service selects 20 frames from the entire video, which might not capture all the critical moments or details. Frame selection can be approximately evenly spread through the video or focused by a specific video retrieval query, depending on the prompt.
+- **Language support**: The service primarily supports English for grounding with transcripts. Transcripts don't provide accurate information on lyrics in songs.
 
 ## Next steps
 

articles/ai-services/policy-reference.md

@@ -1,7 +1,7 @@
 ---
 title: Built-in policy definitions for Azure AI services
 description: Lists Azure Policy built-in policy definitions for Azure AI services. These built-in policy definitions provide common approaches to managing your Azure resources.
-ms.date: 12/13/2023
+ms.date: 01/02/2024
 author: nitinme
 ms.author: nitinme
 ms.service: azure-ai-services

articles/aks/policy-reference.md

@@ -1,7 +1,7 @@
 ---
 title: Built-in policy definitions for Azure Kubernetes Service
 description: Lists Azure Policy built-in policy definitions for Azure Kubernetes Service. These built-in policy definitions provide common approaches to managing your Azure resources.
-ms.date: 12/13/2023
+ms.date: 01/02/2024
 ms.topic: reference
 ms.custom: subject-policy-reference
 ---

articles/api-management/policy-reference.md

@@ -1,7 +1,7 @@
 ---
 title: Built-in policy definitions for Azure API Management
 description: Lists Azure Policy built-in policy definitions for Azure API Management. These built-in policy definitions provide approaches to managing your Azure resources.
-ms.date: 12/13/2023
+ms.date: 01/02/2024
 author: dlepow
 ms.author: danlep
 ms.service: api-management

articles/app-service/policy-reference.md

@@ -1,7 +1,7 @@
 ---
 title: Built-in policy definitions for Azure App Service
 description: Lists Azure Policy built-in policy definitions for Azure App Service. These built-in policy definitions provide common approaches to managing your Azure resources.
-ms.date: 12/13/2023
+ms.date: 01/02/2024
 ms.topic: reference
 ms.custom: subject-policy-reference
 author: cephalin

articles/attestation/policy-reference.md

@@ -1,7 +1,7 @@
 ---
 title: Built-in policy definitions for Azure Attestation
 description: Lists Azure Policy built-in policy definitions for Azure Attestation. These built-in policy definitions provide common approaches to managing your Azure resources.
-ms.date: 12/13/2023
+ms.date: 01/02/2024
 ms.topic: reference
 ms.service: attestation
 author: msmbaldwin

articles/automation/policy-reference.md

@@ -1,7 +1,7 @@
 ---
 title: Built-in policy definitions for Azure Automation
 description: Lists Azure Policy built-in policy definitions for Azure Automation. These built-in policy definitions provide common approaches to managing your Azure resources.
-ms.date: 12/13/2023
+ms.date: 01/02/2024
 ms.topic: reference
 ms.custom: subject-policy-reference
 ---

articles/azure-app-configuration/policy-reference.md

@@ -1,7 +1,7 @@
 ---
 title: Built-in policy definitions for Azure App Configuration
 description: Lists Azure Policy built-in policy definitions for Azure App Configuration. These built-in policy definitions provide common approaches to managing your Azure resources.
-ms.date: 12/13/2023
+ms.date: 01/02/2024
 ms.topic: reference
 author: maud-lv
 ms.author: malev

articles/azure-arc/kubernetes/policy-reference.md

@@ -1,7 +1,7 @@
 ---
 title: Built-in policy definitions for Azure Arc-enabled Kubernetes
 description: Lists Azure Policy built-in policy definitions for Azure Arc-enabled Kubernetes. These built-in policy definitions provide common approaches to managing your Azure resources.
-ms.date: 12/13/2023
+ms.date: 01/02/2024
 ms.service: azure-arc
 #ms.subservice: azure-arc-kubernetes coming soon
 ms.topic: reference

articles/azure-arc/servers/policy-reference.md

@@ -1,7 +1,7 @@
 ---
 title: Built-in policy definitions for Azure Arc-enabled servers
 description: Lists Azure Policy built-in policy definitions for Azure Arc-enabled servers (preview). These built-in policy definitions provide common approaches to managing your Azure resources.
-ms.date: 12/13/2023
+ms.date: 01/02/2024
 ms.topic: reference
 ms.custom: subject-policy-reference
 ---