You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: pages/generative-apis/how-to/use-batch-processing.mdx
+32-46Lines changed: 32 additions & 46 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -21,26 +21,26 @@ Batch processing is designed for:
21
21
<Requirements />
22
22
23
23
- A Scaleway account logged into the [console](https://console.scaleway.com)
24
-
-[Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
24
+
-[Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization. These IAM permissions are `GenerativeApisFullAccess`, `ObjectStorageObjectsRead`,`ObjectStorageObjectsWrite`.
25
25
- A valid [API key](/iam/how-to/create-api-keys/) for API authentication
26
26
- Python 3.7+ installed on your system
27
27
- If bucket policies exists in the bucket storing your `.jsonl` file, the following bucket policies actions `s3:GetObject` and `s3:PutObject` for `scw-managed-genapi-batch` IAM application. This application is auto-generated by Scaleway after your first batch creation.
28
28
29
29
## How batch processing works
30
30
31
-
1. Upload a JSONL input file to an Object Storage bucket.
31
+
1. Upload a JSONL input file to an Object Storage bucket. This file contains all API queries to perform.
32
32
2. Create a batch job referencing this file.
33
33
3. The service processes each line as an individual request.
34
-
4. The results are written to an output file in the same bucket, and named `{filename}-output.jsonl` and `{filename}-error.jsonl`
34
+
4. The results are written to an output file in the same bucket, and named `{filename}_output.jsonl` and `{filename}_error.jsonl`
35
35
5. Retrieve the output file once the job status is `completed`.
36
36
37
37
Each line of the input file must be a valid JSON object representing a request to the Generative API.
38
38
39
39
Example `input.jsonl`:
40
40
41
41
```json
42
-
{"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "voxtral-small-24b-2507", "messages": [{"role": "system", "content": "You are a helpful assistant."},{"role": "user", "content": "Translate this into French: Hello world!"}],"max_completion_tokens": 500}}
43
-
{"custom_id": "request-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "voxtral-small-24b-2507", "messages": [{"role": "system", "content": "You are an unhelpful assistant."},{"role": "user", "content": "Write a poem about the ocean."}],"max_completion_tokens": 500}}
42
+
{"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "mistral-small-3.2-24b-instruct-2506", "messages": [{"role": "system", "content": "You are a helpful assistant."},{"role": "user", "content": "Translate this into French: Hello world!"}],"max_completion_tokens": 500}}
43
+
{"custom_id": "request-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "mistral-small-3.2-24b-instruct-2506", "messages": [{"role": "system", "content": "You are an unhelpful assistant."},{"role": "user", "content": "Write a poem about the ocean."}],"max_completion_tokens": 500}}
44
44
```
45
45
46
46
<Messagetype="note">
@@ -51,7 +51,7 @@ Example `input.jsonl`:
51
51
52
52
Batch processing relies on an Object Storage bucket. You can use any of your existing Object Storage buckets for batch processing or create a new one.
53
53
54
-
To use your bucket with batch procession, you must configure a bucket policy allowing the Generative APIs application principal to:
54
+
If your bucket has at least one [bucket policy](object-storage/api-cli/bucket-policy/) configured, you must edit this bucket policy or create a new one allowing the Scaleway-managed application principal (named `scw-managed-genapi-batch`) to perform the following actions:
55
55
56
56
*`s3:GetObject`
57
57
*`s3:PutObject`
@@ -67,7 +67,7 @@ Below is an example bucket policy:
@@ -90,7 +90,9 @@ Below is an example bucket policy:
90
90
]
91
91
}
92
92
```
93
-
93
+
Note that to configure this bucket policy, you need to replace:
94
+
-`user_id:YOUR_IAM_USER_ID` with your IAM user id
95
+
-`application_id:SCW_MANAGED_GENAPI_BATCH_ID` with IAM application id from the application named `scw-managed-genapi-batch`. Note that this application (and its corresponding id) will be created automatically in your organization after you initiate a first batch query. If you have bucket policies configured, this means you need to create a first batch to obtain `scw-managed-genapi-batch` IAM id (even if this first batch fails).
94
96
<Messagetype="tip">
95
97
The `Scaleway secure statement` ensures that you retain full access to your bucket and do not accidentally lock yourself out.
96
98
</Message>
@@ -141,8 +143,8 @@ Each line of the output file corresponds to one input request.
141
143
Example:
142
144
143
145
```json
144
-
{"id":"93807e44-09fa-4a9f-baa3-574164651535","custom_id":"request-1","error":null,"response":{"request_id":"7a38ba8c-0fad-4923-9214-9b218a416b50","status_code":200,"body":{"id":"chatcmpl-7a38ba8c-0fad-4923-9214-9b218a416b50","object":"chat.completion","created":1771335694,"model":"voxtral-small-24b-2507","choices":[{"index":0,"message":{"role":"assistant","content":"Bonjour le monde!","refusal":null,"annotations":null,"audio":null,"function_call":null,"tool_calls":[],"reasoning_content":null},"logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":19,"total_tokens":25,"completion_tokens":6,"prompt_tokens_details":null},"prompt_logprobs":null,"prompt_token_ids":null,"kv_transfer_params":null}}}
145
-
{"id":"93807e44-09fa-4a9f-baa3-574164651535","custom_id":"request-2","error":null,"response":{"request_id":"01c2cc2c-b072-45fb-9031-2b10b75f1b2f","status_code":200,"body":{"id":"chatcmpl-01c2cc2c-b072-45fb-9031-2b10b75f1b2f","object":"chat.completion","created":1771335694,"model":"voxtral-small-24b-2507","choices":[{"index":0,"message":{"role":"assistant","content":"Oh, the ocean, so vast and so wide,\nA shimmering expanse of blue tide.\nIt whispers and roars, in a rhythm so grand,\nA symphony of waves, in the ocean's band.\n\nIt's a mystery, deep and profound,\nWith secrets untold, in its watery ground.\nCreatures unseen, in its depths they reside,\nIn the ocean's embrace, they confide.\n\nIt's a force to be reckoned with, a power so great,\nA tempestuous beast, that can't be tamed, can't be sate.\nYet, it's also a cradle, a gentle, soothing song,\nA lullaby sung, to the weary and strong.\n\nSo here's to the ocean, in all its might and grace,\nA wonder of nature, in its own special place.\nMay it continue to inspire, to captivate and to amaze,\nThe ocean, the ocean, in all its ways.","refusal":null,"annotations":null,"audio":null,"function_call":null,"tool_calls":[],"reasoning_content":null},"logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":20,"total_tokens":218,"completion_tokens":198,"prompt_tokens_details":null},"prompt_logprobs":null,"prompt_token_ids":null,"kv_transfer_params":null}}}
146
+
{"id":"93807e44-09fa-4a9f-baa3-574164651535","custom_id":"request-1","error":null,"response":{"request_id":"7a38ba8c-0fad-4923-9214-9b218a416b50","status_code":200,"body":{"id":"chatcmpl-7a38ba8c-0fad-4923-9214-9b218a416b50","object":"chat.completion","created":1771335694,"model":"mistral-small-3.2-24b-instruct-2506","choices":[{"index":0,"message":{"role":"assistant","content":"Bonjour le monde!","refusal":null,"annotations":null,"audio":null,"function_call":null,"tool_calls":[],"reasoning_content":null},"logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":19,"total_tokens":25,"completion_tokens":6,"prompt_tokens_details":null},"prompt_logprobs":null,"prompt_token_ids":null,"kv_transfer_params":null}}}
147
+
{"id":"93807e44-09fa-4a9f-baa3-574164651535","custom_id":"request-2","error":null,"response":{"request_id":"01c2cc2c-b072-45fb-9031-2b10b75f1b2f","status_code":200,"body":{"id":"chatcmpl-01c2cc2c-b072-45fb-9031-2b10b75f1b2f","object":"chat.completion","created":1771335694,"model":"mistral-small-3.2-24b-instruct-2506","choices":[{"index":0,"message":{"role":"assistant","content":"Oh, the ocean, so vast and so wide,\nA shimmering expanse of blue tide.\nIt whispers and roars, in a rhythm so grand,\nA symphony of waves, in the ocean's band.\n\nIt's a mystery, deep and profound,\nWith secrets untold, in its watery ground.\nCreatures unseen, in its depths they reside,\nIn the ocean's embrace, they confide.\n\nIt's a force to be reckoned with, a power so great,\nA tempestuous beast, that can't be tamed, can't be sate.\nYet, it's also a cradle, a gentle, soothing song,\nA lullaby sung, to the weary and strong.\n\nSo here's to the ocean, in all its might and grace,\nA wonder of nature, in its own special place.\nMay it continue to inspire, to captivate and to amaze,\nThe ocean, the ocean, in all its ways.","refusal":null,"annotations":null,"audio":null,"function_call":null,"tool_calls":[],"reasoning_content":null},"logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":20,"total_tokens":218,"completion_tokens":198,"prompt_tokens_details":null},"prompt_logprobs":null,"prompt_token_ids":null,"kv_transfer_params":null}}}
0 commit comments