|
| 1 | +--- |
| 2 | +title: How to use batch processing |
| 3 | +description: Learn how to submit large volumes of requests to Generative APIs asynchronously. |
| 4 | +tags: generative-apis ai-data batch-processing |
| 5 | +dates: |
| 6 | + validation: 2026-02-17 |
| 7 | + posted: 2026-02-17 |
| 8 | +--- |
| 9 | +import Requirements from '@macros/iam/requirements.mdx' |
| 10 | + |
| 11 | +Batch processing allows you to submit large volumes of requests to Generative APIs asynchronously, at a discounted price. |
| 12 | +Instead of sending individual requests, you upload an input file to Object Storage and create a batch job. The service processes the requests in the background and writes the results to an output file. |
| 13 | + |
| 14 | +<Requirements /> |
| 15 | + |
| 16 | +- A Scaleway account logged into the [console](https://console.scaleway.com) |
| 17 | +- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization. These IAM permissions are `GenerativeApisFullAccess`, `ObjectStorageObjectsRead`,`ObjectStorageObjectsWrite`. |
| 18 | +- A valid [API key](/iam/how-to/create-api-keys/) for API authentication |
| 19 | +- An [Object Storage bucket](/object-storage/how-to/create-a-bucket/) bucket in the same Project in which you want to run the batch job |
| 20 | +- Python 3.7+ installed on your system |
| 21 | +- Policies allowing the Scaleway-managed application principal `scw-managed-genapi-batch` to perform the `s3:GetObject` and `s3:PutObject` actions in your bucket. This application is auto-generated by Scaleway after you create your first batch. |
| 22 | + |
| 23 | +## How batch processing works |
| 24 | + |
| 25 | +1. Upload a JSONL input file to an Object Storage bucket. This file contains all API queries to perform. |
| 26 | +2. Create a batch job referencing this file. |
| 27 | +3. The service processes each line as an individual request. |
| 28 | +4. The results are written to an output file in the same bucket, and named `{filename}_output.jsonl` and `{filename}_error.jsonl`. |
| 29 | +5. Retrieve the output file once the job status is `completed`. |
| 30 | + |
| 31 | +Each line of the input file must be a valid JSON object representing a request to the Generative API. |
| 32 | + |
| 33 | +Example `input.jsonl`: |
| 34 | + |
| 35 | +<Message type="note"> |
| 36 | + The JSONL file must adhere to the following requirements: |
| 37 | + * The `custom_id` field must be unique for each request. |
| 38 | + * The `method`, `url`, and `model` fields must remain consistent across all requests. |
| 39 | + * The `body` object must comply with the [Generative API documentation](https://www.scaleway.com/en/developers/api/generative-apis/). |
| 40 | +</Message> |
| 41 | + |
| 42 | +```json |
| 43 | +{"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "mistral-small-3.2-24b-instruct-2506", "messages": [{"role": "system", "content": "You are a helpful assistant."},{"role": "user", "content": "Translate this into French: Hello world!"}],"max_completion_tokens": 500}} |
| 44 | +{"custom_id": "request-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "mistral-small-3.2-24b-instruct-2506", "messages": [{"role": "system", "content": "You are an unhelpful assistant."},{"role": "user", "content": "Write a poem about the ocean."}],"max_completion_tokens": 500}} |
| 45 | +``` |
| 46 | + |
| 47 | +<Message type="note"> |
| 48 | + Batch processing can take up to 24 hours. After this timeframe, the batch job will be terminated. Any remaining requests will not be processed. You will only be billed for processed requests. |
| 49 | +</Message> |
| 50 | + |
| 51 | +### Object Storage bucket permissions |
| 52 | + |
| 53 | +Batch processing relies on an Object Storage bucket. You can use any of your existing Object Storage buckets for batch processing or create a new one. |
| 54 | + |
| 55 | +If your bucket has at least one [bucket policy](object-storage/api-cli/bucket-policy/) configured, you must edit this bucket policy or create a new one allowing the Scaleway-managed application principal (named `scw-managed-genapi-batch`) to perform the following actions: |
| 56 | + |
| 57 | +* `s3:GetObject` |
| 58 | +* `s3:PutObject` |
| 59 | + |
| 60 | +Below is an example bucket policy: |
| 61 | + |
| 62 | +```json |
| 63 | +{ |
| 64 | + "Version": "2023-04-17", |
| 65 | + "Id": "allow-only-bucket", |
| 66 | + "Statement": [ |
| 67 | + { |
| 68 | + "Sid": "scw-managed", |
| 69 | + "Effect": "Allow", |
| 70 | + "Principal": { |
| 71 | + "SCW": "application_id:example-4096-adea-b6e03b44a99d" |
| 72 | + }, |
| 73 | + "Action": [ |
| 74 | + "s3:GetObject", |
| 75 | + "s3:PutObject" |
| 76 | + ], |
| 77 | + "Resource": "my-readonly-bucket/*" |
| 78 | + }, |
| 79 | + { |
| 80 | + "Sid": "Scaleway secure statement", |
| 81 | + "Effect": "Allow", |
| 82 | + "Principal": { |
| 83 | + "SCW": "user_id:example-9f9c-46b1-b2e0-ed883a6c54ce" |
| 84 | + }, |
| 85 | + "Action": "*", |
| 86 | + "Resource": [ |
| 87 | + "my-readonly-bucket", |
| 88 | + "my-readonly-bucket/*" |
| 89 | + ] |
| 90 | + } |
| 91 | + ] |
| 92 | +} |
| 93 | +``` |
| 94 | +Note that to configure this bucket policy, you must replace: |
| 95 | +- `user_id:YOUR_IAM_USER_ID` with your IAM user ID |
| 96 | +- `application_id:SCW_MANAGED_GENAPI_BATCH_ID` with the ID of the IAM application named `scw-managed-genapi-batch`. This application and its corresponding ID will be created automatically in your Organization after you initiate a first batch query. If you have bucket policies configured, you must create a first batch to obtain the `scw-managed-genapi-batch` IAM ID, even if the first batch fails. |
| 97 | +<Message type="tip"> |
| 98 | + The `Scaleway secure statement` ensures that you retain full access to your bucket and do not accidentally lock yourself out. |
| 99 | +</Message> |
| 100 | + |
| 101 | + |
| 102 | +## Creating a batch using the console |
| 103 | + |
| 104 | +### Creating the batch job |
| 105 | + |
| 106 | +1. Click **Generative APIs** on the **AI** section of the side menu. The Generative APIs overview page appears. |
| 107 | +2. Click the **Batches** tab. |
| 108 | +3. Click **Create batch**. The batch creation wizard opens. |
| 109 | + |
| 110 | + <Message type="important"> |
| 111 | + The Object Storage bucket and the batch job must belong to the same Project. |
| 112 | + </Message> |
| 113 | + |
| 114 | +4. Provide the following: |
| 115 | + * Input file path (e.g., `s3://scw-managed-genapi-batch/input.jsonl`) |
| 116 | + <Message type="tip"> |
| 117 | + You can upload your `input.jsonl` file directly by clicking **+ Add file**. |
| 118 | + </Message> |
| 119 | + |
| 120 | + * Output prefix (optional) |
| 121 | + |
| 122 | +5. Click **Create batch**. The batch job enters the `Validating` state. |
| 123 | + |
| 124 | +### Monitoring the job |
| 125 | + |
| 126 | +You can monitor: |
| 127 | + |
| 128 | +* Status (`Validating`, `In progress`, `Completed`, `Failed`) |
| 129 | +* Number of processed requests |
| 130 | +* Error count |
| 131 | + |
| 132 | +Once completed, you can download the generated output file from Object Storage. |
| 133 | + |
| 134 | +### Output format |
| 135 | + |
| 136 | +Each line of the output file corresponds to one input request. |
| 137 | + |
| 138 | +Example: |
| 139 | + |
| 140 | +```json |
| 141 | +{"id":"93807e44-09fa-4a9f-baa3-574164651535","custom_id":"request-1","error":null,"response":{"request_id":"7a38ba8c-0fad-4923-9214-9b218a416b50","status_code":200,"body":{"id":"chatcmpl-7a38ba8c-0fad-4923-9214-9b218a416b50","object":"chat.completion","created":1771335694,"model":"mistral-small-3.2-24b-instruct-2506","choices":[{"index":0,"message":{"role":"assistant","content":"Bonjour le monde!","refusal":null,"annotations":null,"audio":null,"function_call":null,"tool_calls":[],"reasoning_content":null},"logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":19,"total_tokens":25,"completion_tokens":6,"prompt_tokens_details":null},"prompt_logprobs":null,"prompt_token_ids":null,"kv_transfer_params":null}}} |
| 142 | +{"id":"93807e44-09fa-4a9f-baa3-574164651535","custom_id":"request-2","error":null,"response":{"request_id":"01c2cc2c-b072-45fb-9031-2b10b75f1b2f","status_code":200,"body":{"id":"chatcmpl-01c2cc2c-b072-45fb-9031-2b10b75f1b2f","object":"chat.completion","created":1771335694,"model":"mistral-small-3.2-24b-instruct-2506","choices":[{"index":0,"message":{"role":"assistant","content":"Oh, the ocean, so vast and so wide,\nA shimmering expanse of blue tide.\nIt whispers and roars, in a rhythm so grand,\nA symphony of waves, in the ocean's band.\n\nIt's a mystery, deep and profound,\nWith secrets untold, in its watery ground.\nCreatures unseen, in its depths they reside,\nIn the ocean's embrace, they confide.\n\nIt's a force to be reckoned with, a power so great,\nA tempestuous beast, that can't be tamed, can't be sate.\nYet, it's also a cradle, a gentle, soothing song,\nA lullaby sung, to the weary and strong.\n\nSo here's to the ocean, in all its might and grace,\nA wonder of nature, in its own special place.\nMay it continue to inspire, to captivate and to amaze,\nThe ocean, the ocean, in all its ways.","refusal":null,"annotations":null,"audio":null,"function_call":null,"tool_calls":[],"reasoning_content":null},"logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":20,"total_tokens":218,"completion_tokens":198,"prompt_tokens_details":null},"prompt_logprobs":null,"prompt_token_ids":null,"kv_transfer_params":null}}} |
| 143 | +``` |
| 144 | + |
| 145 | +## Creating a batch using the OpenAI Python SDK |
| 146 | + |
| 147 | +You can also create and manage batches programmatically using the OpenAI-compatible SDK. |
| 148 | + |
| 149 | +```py |
| 150 | +import os |
| 151 | +import time |
| 152 | +import logging |
| 153 | +from openai import OpenAI |
| 154 | + |
| 155 | +# Initialize client with staging base URL |
| 156 | +client = OpenAI( |
| 157 | + api_key=os.getenv("SCW_SECRET_KEY"), |
| 158 | + base_url="https://api.scaleway.ai/v1" # No "/batches" needed |
| 159 | +) |
| 160 | + |
| 161 | +# Your S3 input file URL (only path to the file without query parameters, hence not a pre-signed URL) |
| 162 | +input_file_url = "https://<my-bucket-name>.s3.<region>.scw.cloud/input.jsonl" |
| 163 | + |
| 164 | +# Submit the batch job |
| 165 | +print("Submitting batch job...") |
| 166 | +batch = client.batches.create( |
| 167 | + input_file_id=input_file_url, |
| 168 | + endpoint="/v1/chat/completions", |
| 169 | + completion_window="24h" |
| 170 | +) |
| 171 | + |
| 172 | +print(f"Batch created: {batch.id}") |
| 173 | +print(f"Initial status: {batch.status}") |
| 174 | + |
| 175 | +# Polling function |
| 176 | +def wait_for_completion(batch_id, check_interval=10): |
| 177 | + print(f"Polling every {check_interval}s for completion...\n") |
| 178 | + |
| 179 | + while True: |
| 180 | + batch_job = client.batches.retrieve(batch_id) |
| 181 | + last_update_timestamp = batch_job.completed_at or batch_job.in_progress_at or batch_job.created_at |
| 182 | + print(f"Status: {batch_job.status.upper()} (Updated: {time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(last_update_timestamp))})") |
| 183 | + |
| 184 | + if batch_job.status == "completed": |
| 185 | + print(f"\nBatch completed successfully!") |
| 186 | + print(f"Download results: {batch_job.output_file_id}") |
| 187 | + return batch_job.output_file_id |
| 188 | + |
| 189 | + elif batch_job.status == "failed": |
| 190 | + if hasattr(batch_job, 'errors') and batch_job.errors: |
| 191 | + print(f"Batch failed: {batch_job.errors}") |
| 192 | + else: |
| 193 | + print(f"Batch failed: Unknown error") |
| 194 | + return None |
| 195 | + |
| 196 | + elif batch_job.status in ["in_progress", "validating"]: |
| 197 | + time.sleep(check_interval) |
| 198 | + else: |
| 199 | + print(f"Unexpected status: {batch_job.status}") |
| 200 | + return None |
| 201 | + |
| 202 | +# Start polling |
| 203 | +wait_for_completion(batch.id) |
0 commit comments