Skip to content

Commit d23a81f

Browse files
mrbullwinklemrbullwinkle
committed
update
1 parent fc33129 commit d23a81f

File tree

3 files changed

+254
-11
lines changed

3 files changed

+254
-11
lines changed

‎articles/ai-services/openai/how-to/batch.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ Refer to the [models page](../concepts/models.md) for the most up-to-date inform
6767

6868
### API support
6969

70-
API support was first added with `2024-07-01-preview`.
70+
API support was first added with `2024-07-01-preview`. Use `2024-10-01-preview` to take advantage of the latest features.
7171

7272
### Not supported
7373

@@ -83,7 +83,7 @@ In the Studio UI the deployment type will appear as `Global-Batch`.
8383
:::image type="content" source="../media/how-to/global-batch/global-batch.png" alt-text="Screenshot that shows the model deployment dialog in Azure OpenAI Studio with Global-Batch deployment type highlighted." lightbox="../media/how-to/global-batch/global-batch.png":::
8484

8585
> [!TIP]
86-
> Each line of your input file for batch processing has a `model` attribute that requires a global batch **deployment name**. For a given input file, all names must be the same deployment name. This is different from OpenAI where the concept of model deployments does not exist.
86+
> Each line of your input file for batch processing has a `model` attribute that requires a global batch **deployment name**. For a given input file, all names must be the same deployment name. This is different from OpenAI where the concept of model deployments does not exist.
8787
>
8888
> For the best performance we recommend submitting large files for batch processing, rather than a large number of small files with only a few lines in each file.
8989

‎articles/ai-services/openai/includes/batch/batch-python.md‎

Lines changed: 234 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ description: Azure OpenAI model global batch Python
55
manager: nitinme
66
ms.service: azure-ai-openai
77
ms.topic: include
8-
ms.date: 07/22/2024
8+
ms.date: 10/15/2024
99
---
1010

1111
## Prerequisites
@@ -74,13 +74,42 @@ Once your input file is prepared, you first need to upload the file to then be a
7474

7575
[!INCLUDE [Azure key vault](~/reusable-content/ce-skilling/azure/includes/ai-services/security/azure-key-vault.md)]
7676

77+
# [Python (Microsoft Entra ID)](#tab/python-secure)
78+
79+
```python
80+
import os
81+
from openai import AzureOpenAI
82+
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
83+
84+
token_provider = get_bearer_token_provider(
85+
DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default"
86+
)
87+
88+
client = AzureOpenAI(
89+
azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"),
90+
azure_ad_token_provider=token_provider,
91+
api_version="2024-10-01-preview"
92+
)
93+
94+
# Upload a file with a purpose of "batch"
95+
file = client.files.create(
96+
file=open("test.jsonl", "rb"),
97+
purpose="batch"
98+
)
99+
100+
print(file.model_dump_json(indent=2))
101+
file_id = file.id
102+
```
103+
104+
# [Python (API Key)](#tab/python-key)
105+
77106
```python
78107
import os
79108
from openai import AzureOpenAI
80109

81110
client = AzureOpenAI(
82111
api_key=os.getenv("AZURE_OPENAI_API_KEY"),
83-
api_version="2024-07-01-preview",
112+
api_version="2024-10-01-preview",
84113
azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT")
85114
)
86115

@@ -94,6 +123,8 @@ print(file.model_dump_json(indent=2))
94123
file_id = file.id
95124
```
96125

126+
---
127+
97128
**Output:**
98129

99130
```json
@@ -367,3 +398,204 @@ List all batch jobs for a particular Azure OpenAI resource.
367398
```python
368399
client.batches.list()
369400
```
401+
402+
### List batch (Preview)
403+
404+
Use the REST API to list all batch jobs with additional sorting/filtering options.
405+
406+
In the examples below we are providing the `generate_time_filter` function to make constructing the filter easier. If you don't wish to use this function the format of the filter string would look like `created_at gt 1728773533 and created_at lt 1729032733 and status eq 'Completed'`.
407+
408+
# [Python (Microsoft Entra ID)](#tab/python-secure)
409+
410+
```python
411+
import requests
412+
import json
413+
from datetime import datetime, timedelta
414+
from azure.identity import DefaultAzureCredential
415+
416+
token_credential = DefaultAzureCredential()
417+
token = token_credential.get_token('https://cognitiveservices.azure.com/.default')
418+
419+
endpoint = "https://{YOUR_RESOURCE_NAME}.openai.azure.com/"
420+
api_version = "2024-10-01-preview"
421+
url = f"{endpoint}openai/batches"
422+
order = "created_at asc"
423+
time_filter = lambda: generate_time_filter("past 8 hours")
424+
425+
# Additional filter examples:
426+
#time_filter = lambda: generate_time_filter("past 1 day")
427+
#time_filter = lambda: generate_time_filter("past 3 days", status="Completed")
428+
429+
def generate_time_filter(time_range, status=None):
430+
now = datetime.now()
431+
432+
if 'day' in time_range:
433+
days = int(time_range.split()[1])
434+
start_time = now - timedelta(days=days)
435+
elif 'hour' in time_range:
436+
hours = int(time_range.split()[1])
437+
start_time = now - timedelta(hours=hours)
438+
else:
439+
raise ValueError("Invalid time range format. Use 'past X day(s)' or 'past X hour(s)'")
440+
441+
start_timestamp = int(start_time.timestamp())
442+
end_timestamp = int(now.timestamp())
443+
444+
filter_string = f"created_at gt {start_timestamp} and created_at lt {end_timestamp}"
445+
446+
if status:
447+
filter_string += f" and status eq '{status}'"
448+
449+
return filter_string
450+
451+
filter = time_filter()
452+
453+
headers = {'Authorization': 'Bearer ' + token.token}
454+
455+
params = {
456+
"api-version": api_version,
457+
"$filter": filter,
458+
"$orderby": order
459+
}
460+
461+
response = requests.get(url, headers=headers, params=params)
462+
463+
json_data = response.json()
464+
465+
if response.status_code == 200:
466+
print(json.dumps(json_data, indent=2))
467+
else:
468+
print(f"Request failed with status code: {response.status_code}")
469+
print(response.text)
470+
```
471+
472+
# [Python (API Key)](#tab/python-key)
473+
474+
```python
475+
import os
476+
import requests
477+
import json
478+
from datetime import datetime, timedelta
479+
480+
api_key = os.getenv("AZURE_OPENAI_API_KEY"),
481+
endpoint = os.getenv("AZURE_OPENAI_ENDPOINT")
482+
api_version = "2024-10-01-preview"
483+
url = f"{endpoint}openai/batches"
484+
order = "created_at asc"
485+
486+
time_filter = lambda: generate_time_filter("past 8 hours")
487+
488+
# Additional filter examples:
489+
#time_filter = lambda: generate_time_filter("past 1 day")
490+
#time_filter = lambda: generate_time_filter("past 3 days", status="Completed")
491+
492+
def generate_time_filter(time_range, status=None):
493+
now = datetime.now()
494+
495+
if 'day' in time_range:
496+
days = int(time_range.split()[1])
497+
start_time = now - timedelta(days=days)
498+
elif 'hour' in time_range:
499+
hours = int(time_range.split()[1])
500+
start_time = now - timedelta(hours=hours)
501+
else:
502+
raise ValueError("Invalid time range format. Use 'past X day(s)' or 'past X hour(s)'")
503+
504+
start_timestamp = int(start_time.timestamp())
505+
end_timestamp = int(now.timestamp())
506+
507+
filter_string = f"created_at gt {start_timestamp} and created_at lt {end_timestamp}"
508+
509+
if status:
510+
filter_string += f" and status eq '{status}'"
511+
512+
return filter_string
513+
514+
filter = time_filter()
515+
516+
headers = {
517+
"api-key": api_key
518+
}
519+
520+
params = {
521+
"api-version": api_version,
522+
"$filter": filter,
523+
"$orderby": order
524+
}
525+
526+
response = requests.get(url, headers=headers, params=params)
527+
528+
json_data = response.json()
529+
530+
if response.status_code == 200:
531+
print(json.dumps(json_data, indent=2))
532+
else:
533+
print(f"Request failed with status code: {response.status_code}")
534+
print(response.text)
535+
```
536+
537+
---
538+
539+
**Output:**
540+
541+
```output
542+
{
543+
"data": [
544+
{
545+
"cancelled_at": null,
546+
"cancelling_at": null,
547+
"completed_at": 1729011896,
548+
"completion_window": "24h",
549+
"created_at": 1729011128,
550+
"error_file_id": "file-472c0626-4561-4327-9e4e-f41afbfb30e6",
551+
"expired_at": null,
552+
"expires_at": 1729097528,
553+
"failed_at": null,
554+
"finalizing_at": 1729011805,
555+
"id": "batch_4ddc7b60-19a9-419b-8b93-b9a3274b33b5",
556+
"in_progress_at": 1729011493,
557+
"input_file_id": "file-f89384af0082485da43cb26b49dc25ce",
558+
"errors": null,
559+
"metadata": null,
560+
"object": "batch",
561+
"output_file_id": "file-62bebde8-e767-4cd3-a0a1-28b214dc8974",
562+
"request_counts": {
563+
"total": 3,
564+
"completed": 2,
565+
"failed": 1
566+
},
567+
"status": "completed",
568+
"endpoint": "/chat/completions"
569+
},
570+
{
571+
"cancelled_at": null,
572+
"cancelling_at": null,
573+
"completed_at": 1729016366,
574+
"completion_window": "24h",
575+
"created_at": 1729015829,
576+
"error_file_id": "file-85ae1971-9957-4511-9eb4-4cc9f708b904",
577+
"expired_at": null,
578+
"expires_at": 1729102229,
579+
"failed_at": null,
580+
"finalizing_at": 1729016272,
581+
"id": "batch_6287485f-50fc-4efa-bcc5-b86690037f43",
582+
"in_progress_at": 1729016126,
583+
"input_file_id": "file-686746fcb6bc47f495250191ffa8a28e",
584+
"errors": null,
585+
"metadata": null,
586+
"object": "batch",
587+
"output_file_id": "file-04399828-ae0b-4825-9b49-8976778918cb",
588+
"request_counts": {
589+
"total": 3,
590+
"completed": 2,
591+
"failed": 1
592+
},
593+
"status": "completed",
594+
"endpoint": "/chat/completions"
595+
}
596+
],
597+
"first_id": "batch_4ddc7b60-19a9-419b-8b93-b9a3274b33b5",
598+
"has_more": false,
599+
"last_id": "batch_6287485f-50fc-4efa-bcc5-b86690037f43"
600+
}
601+
```

‎articles/ai-services/openai/includes/batch/batch-rest.md‎

Lines changed: 18 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,7 @@ Once your input file is prepared, you first need to upload the file to then be a
6464
[!INCLUDE [Azure key vault](~/reusable-content/ce-skilling/azure/includes/ai-services/security/azure-key-vault.md)]
6565

6666
```http
67-
curl -X POST https://YOUR_RESOURCE_NAME.openai.azure.com/openai/files?api-version=2024-07-01-preview \
67+
curl -X POST https://YOUR_RESOURCE_NAME.openai.azure.com/openai/files?api-version=2024-10-01-preview \
6868
-H "Content-Type: multipart/form-data" \
6969
-H "api-key: $AZURE_OPENAI_API_KEY" \
7070
-F "purpose=batch" \
@@ -94,7 +94,7 @@ The above code assumes a particular file path for your test.jsonl file. Adjust t
9494
Depending on the size of your upload file it might take some time before it's fully uploaded and processed. To check on your file upload status run:
9595

9696
```http
97-
curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/files/{file-id}?api-version=2024-07-01-preview \
97+
curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/files/{file-id}?api-version=2024-10-01-preview \
9898
-H "api-key: $AZURE_OPENAI_API_KEY"
9999
```
100100

@@ -118,7 +118,7 @@ curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/files/{file-id}?api-vers
118118
Once your file has uploaded successfully by reaching a status of `processed` you can submit the file for batch processing.
119119

120120
```http
121-
curl -X POST https://YOUR_RESOURCE_NAME.openai.azure.com/openai/batches?api-version=2024-07-01-preview \
121+
curl -X POST https://YOUR_RESOURCE_NAME.openai.azure.com/openai/batches?api-version=2024-10-01-preview \
122122
-H "api-key: $AZURE_OPENAI_API_KEY" \
123123
-H "Content-Type: application/json" \
124124
-d '{
@@ -168,7 +168,7 @@ curl -X POST https://YOUR_RESOURCE_NAME.openai.azure.com/openai/batches?api-vers
168168
Once you have created batch job successfully you can monitor its progress either in the Studio or programatically. When checking batch job progress we recommend waiting at least 60 seconds in between each status call.
169169

170170
```http
171-
curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/batches/{batch_id}?api-version=2024-07-01-preview \
171+
curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/batches/{batch_id}?api-version=2024-10-01-preview \
172172
-H "api-key: $AZURE_OPENAI_API_KEY"
173173
```
174174

@@ -220,7 +220,7 @@ The following status values are possible:
220220
## Retrieve batch job output file
221221

222222
```http
223-
curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/files/{output_file_id}/content?api-version=2024-07-01-preview \
223+
curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/files/{output_file_id}/content?api-version=2024-10-01-preview \
224224
-H "api-key: $AZURE_OPENAI_API_KEY" > batch_output.jsonl
225225
```
226226

@@ -231,7 +231,7 @@ curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/files/{output_file_id}/c
231231
Cancels an in-progress batch. The batch will be in status `cancelling` for up to 10 minutes, before changing to `cancelled`, where it will have partial results (if any) available in the output file.
232232

233233
```http
234-
curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/batches/{batch_id}/cancel?api-version=2024-07-01-preview \
234+
curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/batches/{batch_id}/cancel?api-version=2024-10-01-preview \
235235
-H "api-key: $AZURE_OPENAI_API_KEY"
236236
```
237237

@@ -240,6 +240,17 @@ curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/batches/{batch_id}/cance
240240
List all existing batch jobs for a given Azure OpenAI resource.
241241

242242
```http
243-
curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/batches?api-version=2024-07-01-preview \
243+
curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/batches?api-version=2024-10-01-preview \
244244
-H "api-key: $AZURE_OPENAI_API_KEY"
245245
```
246+
247+
### List batch (Preview)
248+
249+
Use the REST API to list all batch jobs with additional sorting/filtering options.
250+
251+
```http
252+
curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/batches?api-version=2024-10-01-preview \
253+
-H "api-key: $AZURE_OPENAI_API_KEY"
254+
-H "filter": "created_at gt 1728773533 and created_at lt 1729032733 and status eq 'Completed'"
255+
-H "orderby": "created_at asc"
256+
```

0 commit comments

Comments
 (0)