Skip to content

Commit dd54e3b

Browse files
gippyJaroslav Hejlek
andauthored
feat: add API documentation for dataset validation (#1079)
Hey all, this is a proposal of how the dataset validation could be handled in API. We have decided, that no matter how many items in a request you send, if at least one is not valid the whole request fails with `400 Bad request`. My proposal is to return our normal API error, but add "data" property to the error object returned from API and in the data return "invalidItems" which will list all validation errors for all invalid items by index in the items array. So for example if you send a request with 1000 items and 2 of them do not have a required field. The response will look like this: ```json { "error": { "type": "schema-validation-error", "message": "Schema validation failed", "data": { "invalidItems": { "4": ["name is required"], "124": ["name is required"], }, } } } ``` --------- Co-authored-by: Jaroslav Hejlek <[email protected]>
1 parent e38d9a3 commit dd54e3b

File tree

1 file changed

+36
-3
lines changed

1 file changed

+36
-3
lines changed

sources/platform/api_v2/api_v2_reference.apib

Lines changed: 36 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3081,6 +3081,9 @@ The POST payload is a JSON object or a JSON array of objects to save into the da
30813081
**IMPORTANT:** The limit of request payload size for the dataset is 5 MB. If the array exceeds the size,
30823082
you'll need to split it into a number of smaller arrays.
30833083

3084+
If the dataset has fields schema defined, the push request can potentialy fail with `400 Bad Request` if any item does not match the schema.
3085+
In such case, nothing will be inserted into the dataset and the response will contain an error message with a list of invalid items and their validation errors.
3086+
30843087
+ Parameters
30853088

30863089
+ token: `soSkq9ekdmfOslopH` (string, required) - API authentication token.
@@ -3099,9 +3102,6 @@ you'll need to split it into a number of smaller arrays.
30993102
},
31003103
{
31013104
"foo": "hotel"
3102-
},
3103-
{
3104-
"foo": "restaurant"
31053105
}
31063106
]
31073107

@@ -3113,6 +3113,39 @@ you'll need to split it into a number of smaller arrays.
31133113

31143114
+ Attributes(object)
31153115

3116+
+ Response 400 (application/json)
3117+
3118+
+ Headers
3119+
3120+
Location: https://api.apify.com/v2/datasets/WkzbQMuFYuamGv3YF/items
3121+
3122+
+ Body
3123+
3124+
{
3125+
error: {
3126+
type: 'schema-validation-error',
3127+
message: 'Schema validation failed',
3128+
data: {
3129+
invalidItems: [
3130+
{
3131+
itemPosition: 0,
3132+
validationErrors: [
3133+
{
3134+
instancePath: 'foo',
3135+
schemaPath: 'foo',
3136+
keyword: 'type',
3137+
params: {
3138+
type: 'string'
3139+
},
3140+
message: 'must be string'
3141+
}
3142+
]
3143+
}
3144+
]
3145+
}
3146+
}
3147+
}
3148+
31163149
# Group Request queues
31173150

31183151
This section describes API endpoints to manage request queues.

0 commit comments

Comments
 (0)