Skip to content

Commit f88b79e

Browse files
mforiTC-MO
authored and
patrikbraborec
committed
docs: Key-value store schema page (#1645)
Adds key-value store schema page, in the same way as the one for datasets (https://docs.apify.com/platform/actors/development/actor-definition/dataset-schema) See the page here: https://pr-1645.preview.docs.apify.com/platform/actors/development/actor-definition/key-value-store-schema --------- Co-authored-by: Michał Olender <[email protected]>
1 parent 2e245a3 commit f88b79e

File tree

3 files changed

+202
-0
lines changed

3 files changed

+202
-0
lines changed

‎sources/platform/actors/development/actor_definition/key_value_store_schema/images/kv-store-schema-example-run.png‎

294 KB
Loading

‎sources/platform/actors/development/actor_definition/key_value_store_schema/images/kv-store-schema-example-storage.png‎

292 KB
Loading

‎sources/platform/actors/development/actor_definition/key_value_store_schema/index.md‎

Lines changed: 202 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,202 @@
1+
---
2+
title: Key-value store schema specification
3+
sidebar_label: Key-value store schema
4+
sidebar_position: 3
5+
description: Learn how to define and present your key-value store schema to organize records into collections.
6+
slug: /actors/development/actor-definition/key-value-store-schema
7+
---
8+
9+
# Key-value Store Schema Specification
10+
11+
**Learn how to define and present your key-value store schema to organize records into collections.**
12+
13+
---
14+
15+
The key‑value store schema organizes keys into logical groups called collections, which can be used to filter and categorize data both in the API and the visual user interface. This organization helps users navigate and find specific data more efficiently, while schema‑defined rules (such as content types and JSON schema) ensure that stored values remain consistent and valid.
16+
17+
## Example
18+
19+
Consider an example Actor that calls `Actor.setValue()` to save a record into the key-value store:
20+
21+
```javascript title="main.js"
22+
import { Actor } from 'apify';
23+
// Initialize the JavaScript SDK
24+
await Actor.init();
25+
26+
/**
27+
* Actor code
28+
*/
29+
await Actor.setValue('document-1', 'my text data', { contentType: 'text/plain' });
30+
31+
// ...
32+
33+
await Actor.setValue(`image-${imageID}`, imageBuffer, { contentType: 'image/jpeg' });
34+
35+
// Exit successfully
36+
await Actor.exit();
37+
```
38+
39+
To configure the key-value store schema, use the following template for the `.actor/actor.json` configuration:
40+
41+
```json title=".actor/actor.json"
42+
{
43+
"actorSpecification": 1,
44+
"name": "Actor Name",
45+
"title": "Actor Title",
46+
"version": "1.0.0",
47+
"storages": {
48+
"keyValueStore": {
49+
"actorKeyValueStoreSchemaVersion": 1,
50+
"title": "Key-Value Store Schema",
51+
"collections": {
52+
"documents": {
53+
"title": "Documents",
54+
"description": "Text documents stored by the Actor.",
55+
"keyPrefix": "document-"
56+
},
57+
"images": {
58+
"title": "Images",
59+
"description": "Images stored by the Actor.",
60+
"keyPrefix": "image-",
61+
"contentTypes": ["image/jpeg"]
62+
}
63+
}
64+
}
65+
}
66+
}
67+
```
68+
69+
The template above defines the configuration for the default key-value store.
70+
Each collection can define its member keys using one of the following properties:
71+
72+
- `keyPrefix` - All keys starting with the specified prefix will be included in the collection (e.g., all keys starting with "document-").
73+
- `key` - A specific individual key that will be included in the collection.
74+
75+
You must use either `key` or `keyPrefix` for each collection, but not both.
76+
77+
Once the schema is defined, tabs for each collection will appear in the **Storage** tab of the Actor's run:
78+
79+
![Storages tab in Run](images/kv-store-schema-example-run.png)
80+
81+
The tabs also appear in the storage detail view:
82+
83+
![Storage detail](images/kv-store-schema-example-storage.png)
84+
85+
### API Example
86+
87+
With the key-value store schema defined, you can use the API to list keys from a specific collection by using the `collection` query parameter when calling the [Get list of keys](https://docs.apify.com/api/v2/key-value-store-keys-get) endpoint:
88+
89+
```http title="Get list of keys from a collection"
90+
GET https://api.apify.com/v2/key-value-stores/{storeId}/keys?collection=documents
91+
```
92+
93+
Example response:
94+
95+
```json
96+
{
97+
"data": {
98+
"items": [
99+
{
100+
"key": "document-1",
101+
"size": 254
102+
},
103+
{
104+
"key": "document-2",
105+
"size": 368
106+
}
107+
],
108+
"count": 2,
109+
"limit": 1000,
110+
"exclusiveStartKey": null,
111+
"isTruncated": false
112+
}
113+
}
114+
```
115+
116+
You can also filter by key prefix using the `prefix` parameter:
117+
118+
```http title="Get list of keys with prefix"
119+
GET https://api.apify.com/v2/key-value-stores/{storeId}/keys?prefix=document-
120+
```
121+
122+
### Schema Validation
123+
124+
When you define a key-value store schema with specific `contentTypes` for collections, the Apify platform validates any data being stored against these specifications. For example, if you've specified that a collection should only contain JSON data with content type `application/json`, attempts to store data with other content types in that collection will be rejected.
125+
126+
The validation happens automatically when you call `Actor.setValue()` or use the [Put record](https://docs.apify.com/api/v2/reference/key-value-stores/record/put-record) API endpoint.
127+
128+
If you've defined a `jsonSchema` for a collection with content type `application/json`, the platform will also validate that the JSON data conforms to the specified schema. This helps ensure data consistency and prevents storing malformed data.
129+
130+
## Structure
131+
132+
Output configuration files need to be located in the `.actor` folder within the Actor's root directory.
133+
134+
You have two choices of how to organize files within the `.actor` folder.
135+
136+
### Single configuration file
137+
138+
```json title=".actor/actor.json"
139+
{
140+
"actorSpecification": 1,
141+
"name": "this-is-book-library-scraper",
142+
"title": "Book Library scraper",
143+
"version": "1.0.0",
144+
"storages": {
145+
"keyValueStore": {
146+
"actorKeyValueStoreSchemaVersion": 1,
147+
"title": "Key-Value Store Schema",
148+
"collections": { /* Define your collections here */ }
149+
}
150+
}
151+
}
152+
```
153+
154+
### Separate configuration files
155+
156+
```json title=".actor/actor.json"
157+
{
158+
"actorSpecification": 1,
159+
"name": "this-is-book-library-scraper",
160+
"title": "Book Library scraper",
161+
"version": "1.0.0",
162+
"storages": {
163+
"keyValueStore": "./key_value_store_schema.json"
164+
}
165+
}
166+
```
167+
168+
```json title=".actor/key_value_store_schema.json"
169+
{
170+
"actorKeyValueStoreSchemaVersion": 1,
171+
"title": "Key-Value Store Schema",
172+
"collections": { /* Define your collections here */ }
173+
}
174+
```
175+
176+
Choose the method that best suits your configuration.
177+
178+
## Key-value store schema structure definitions
179+
180+
The key-value store schema defines the collections of keys and their properties. It allows you to organize and validate data stored by the Actor, making it easier to manage and retrieve specific records.
181+
182+
### Key-value store schema object definition
183+
184+
| Property | Type | Required | Description |
185+
|-----------------------------------|-------------------------------|----------|-----------------------------------------------------------------------------------------------------------------|
186+
| `actorKeyValueStoreSchemaVersion` | integer | true | Specifies the version of key-value store schema structure document. <br/>Currently only version 1 is available. |
187+
| `title` | string | true | Title of the schema |
188+
| `description` | string | false | Description of the schema |
189+
| `collections` | Object | true | An object where each key is a collection ID and its value is a collection definition object (see below). |
190+
191+
### Collection object definition
192+
193+
| Property | Type | Required | Description |
194+
|----------------|--------------|--------------|-------------------------------------------------------------------------------------------------------------------------------------------------|
195+
| `title` | string | true | The collection’s title, shown in the run's storage tab and in the storage detail view, where it appears as a tab for filtering records. |
196+
| `description` | string | false | A description of the collection that appears in UI tooltips. |
197+
| `key` | string | conditional* | Defines a single specific key that will be part of this collection. |
198+
| `keyPrefix` | string | conditional* | Defines a prefix for keys that should be included in this collection. |
199+
| `contentTypes` | string array | false | Allowed content types for records in this collection. Used for validation when storing data. |
200+
| `jsonSchema` | object | false | For collections with content type `application/json`, you can define a JSON schema to validate structure. <br/>Uses JSON Schema Draft 07 format. |
201+
202+
\* Either `key` or `keyPrefix` must be specified for each collection, but not both.

0 commit comments

Comments
 (0)