Skip to content

Commit 9e0c51c

Browse files
committed
update
1 parent 9e5cf33 commit 9e0c51c

File tree

4 files changed

+12
-5
lines changed

4 files changed

+12
-5
lines changed

articles/ai-services/openai/how-to/batch.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -90,11 +90,7 @@ In the Studio UI the deployment type will appear as `Global-Batch`.
9090
:::image type="content" source="../media/how-to/global-batch/global-batch.png" alt-text="Screenshot that shows the model deployment dialog in Azure OpenAI Studio with Global-Batch deployment type highlighted." lightbox="../media/how-to/global-batch/global-batch.png":::
9191

9292
> [!TIP]
93-
> We recommend enabling **dynamic quota** for all global batch model deployments to help avoid job failures due to insufficient enqueued token quota.
94-
>
95-
> Each line of your input file for batch processing has a `model` attribute that requires a global batch **deployment name**. For a given input file, all names must be the same deployment name. This is different from OpenAI where the concept of model deployments does not exist.
96-
>
97-
> For the best performance we recommend submitting large files for batch processing, rather than a large number of small files with only a few lines in each file.
93+
> We recommend enabling **dynamic quota** for all global batch model deployments to help avoid job failures due to insufficient enqueued token quota. Dynamic quota allows your deployment to opportunistically take advantage of more quota when extra capacity is available. When dynamic quota is set to off, your deployment will only be able to process requests up to the enqueued token limit that was defined when you created the deployment.
9894
9995
::: zone pivot="programming-language-ai-studio"
10096

@@ -170,6 +166,10 @@ The `2024-10-01-preview` REST API adds two new response headers:
170166
* `deployment-enqueued-tokens` - A approximate token count for your jsonl file calculated immediately after the batch request is submitted. This value represents an estimate based on the number of characters and is not the true token count.
171167
* `deployment-maximum-enqueued-tokens` The total available enqueued tokens available for this global batch model deployment.
172168

169+
**Example:**
170+
171+
172+
173173
These response headers are only available when making a POST request to begin batch processing of a file with the REST API. The language specific client libraries do not currently return these new response headers.
174174

175175
### What happens if the API doesn't complete my request within the 24 hour time frame?

articles/ai-services/openai/includes/batch/batch-python.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,8 @@ The `custom_id` is required to allow you to identify which individual batch requ
6363

6464
> [!IMPORTANT]
6565
> The `model` attribute must be set to match the name of the Global Batch deployment you wish to target for inference responses. The **same Global Batch model deployment name must be present on each line of the batch file.** If you want to target a different deployment you must do so in a separate batch file/job.
66+
>
67+
> For the best performance we recommend submitting large files for batch processing, rather than a large number of small files with only a few lines in each file.
6668
6769
### Create input file
6870

articles/ai-services/openai/includes/batch/batch-rest.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,9 @@ The `custom_id` is required to allow you to identify which individual batch requ
5252

5353
> [!IMPORTANT]
5454
> The `model` attribute must be set to match the name of the Global Batch deployment you wish to target for inference responses. The **same Global Batch model deployment name must be present on each line of the batch file.** If you want to target a different deployment you must do so in a separate batch file/job.
55+
>
56+
> For the best performance we recommend submitting large files for batch processing, rather than a large number of small files with only a few lines in each file.
57+
5558

5659
### Create input file
5760

articles/ai-services/openai/includes/batch/batch-studio.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,8 @@ The `custom_id` is required to allow you to identify which individual batch requ
5252

5353
> [!IMPORTANT]
5454
> The `model` attribute must be set to match the name of the Global Batch deployment you wish to target for inference responses. The **same Global Batch model deployment name must be present on each line of the batch file.** If you want to target a different deployment you must do so in a separate batch file/job.
55+
>
56+
> For the best performance we recommend submitting large files for batch processing, rather than a large number of small files with only a few lines in each file.
5557
5658
### Create input file
5759

0 commit comments

Comments
 (0)