Skip to content

Commit 2e1555b

Browse files
mforimfori
committed
docs: Key-value store schema page
1 parent 06acc77 commit 2e1555b

File tree

2 files changed

+198
-0
lines changed

2 files changed

+198
-0
lines changed

‎sources/platform/actors/development/actor_definition/key_value_store_schema/images/kv-store-schema-example.png‎

295 KB
Loading

‎sources/platform/actors/development/actor_definition/key_value_store_schema/index.md‎

Lines changed: 198 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,198 @@
1+
---
2+
title: Key-value store schema
3+
sidebar_position: 3
4+
description: Learn how to define and present your key-value store schema in a user-friendly output UI.
5+
slug: /actors/development/actor-definition/key-value-store-schema
6+
---
7+
8+
# Key-value Store Schema Specification
9+
10+
**Learn how to define and present your key-value store schema in a user-friendly UI.**
11+
12+
---
13+
14+
The key-value store schema organizes keys into logical groups called collections, which can be used to filter and categorize data both in the API and the visual user interface. This organization helps users navigate and find specific data more efficiently.
15+
16+
## Example
17+
18+
Let's consider an example Actor that calls `Actor.setValue()` to save a record into the key-value store:
19+
20+
```javascript title="main.js"
21+
import { Actor } from 'apify';
22+
// Initialize the JavaScript SDK
23+
await Actor.init();
24+
25+
/**
26+
* Actor code
27+
*/
28+
await Actor.setValue('document-1', 'my text data', { contentType: 'text/plain' });
29+
30+
// ...
31+
32+
await Actor.setValue(`image-${imageID}`, imageBuffer, { contentType: 'image/jpeg' });
33+
34+
// Exit successfully
35+
await Actor.exit();
36+
```
37+
38+
To set up the key-value store collections using a single configuration file, use the following template for the `.actor/actor.json` configuration:
39+
40+
```json title=".actor/actor.json"
41+
{
42+
"actorSpecification": 1,
43+
"name": "Actor Name",
44+
"title": "Actor Title",
45+
"version": "1.0.0",
46+
"storages": {
47+
"keyValueStore": {
48+
"actorKeyValueStoreSchemaVersion": 1,
49+
"actorSpecification": 1,
50+
"title": "Key-Value Store Schema",
51+
"collections": {
52+
"documents": {
53+
"title": "Documents",
54+
"description": "Text documents stored by the Actor.",
55+
"keyPrefix": "document-"
56+
},
57+
"images": {
58+
"title": "Images",
59+
"description": "Images stored by the Actor.",
60+
"keyPrefix": "image-",
61+
"contentTypes": ["image/jpeg"]
62+
}
63+
}
64+
}
65+
}
66+
}
67+
```
68+
69+
The template above defines the configuration for the default key-value store.
70+
Each collection can define its member keys using one of the following properties:
71+
- `keyPrefix` - All keys starting with the specified prefix will be included in the collection (e.g., all keys starting with "document-").
72+
- `key` - A specific individual key that will be included in the collection (e.g., exactly "summary-data").
73+
74+
Note that you must use either `key` or `keyPrefix` for each collection, but not both.
75+
76+
Under the `Storage` tab property of the run or in the key-value store storage detail, there are the tabs for each collection defined in the configuration:
77+
78+
![Storages tab UI](images/kv-store-schema-example.png)
79+
80+
### API Example
81+
82+
You can use the API to list keys from a specific collection by using the `collection` query parameter when calling the [Get list of keys](https://docs.apify.com/api/v2/key-value-store-keys-get) endpoint:
83+
84+
```http title="Get list of keys from a collection"
85+
GET https://api.apify.com/v2/key-value-stores/{storeId}/keys?collection=documents
86+
```
87+
88+
Example response:
89+
90+
```json
91+
{
92+
"data": {
93+
"items": [
94+
{
95+
"key": "document-1",
96+
"size": 254
97+
},
98+
{
99+
"key": "document-2",
100+
"size": 368
101+
}
102+
],
103+
"count": 2,
104+
"limit": 1000,
105+
"exclusiveStartKey": null,
106+
"isTruncated": false
107+
}
108+
}
109+
```
110+
111+
You can also filter by key prefix using the `prefix` parameter:
112+
113+
```http title="Get list of keys with prefix"
114+
GET https://api.apify.com/v2/key-value-stores/{storeId}/keys?prefix=document-
115+
```
116+
117+
### Schema Validation
118+
119+
When you define a key-value store schema with specific `contentTypes` for collections, the Apify platform validates any data being stored against these specifications. For example, if you've specified that a collection should only contain JSON data with content type `application/json`, attempts to store data with other content types in that collection will be rejected.
120+
121+
The validation happens automatically when you call `Actor.setValue()` or use the [Put record](https://docs.apify.com/api/v2/reference/key-value-stores/record/put-record) API endpoint.
122+
123+
If you've defined a `jsonSchema` for a collection with content type `application/json`, the platform will also validate that the JSON data conforms to the specified schema. This helps ensure data consistency and prevents storing malformed data.
124+
125+
## Structure
126+
127+
Output configuration files need to be located in the `.actor` folder within the Actor's root directory.
128+
129+
You have two choices of how to organize files within the `.actor` folder.
130+
131+
### Single configuration file
132+
133+
```json title=".actor/actor.json"
134+
{
135+
"actorSpecification": 1,
136+
"name": "this-is-book-library-scraper",
137+
"title": "Book Library scraper",
138+
"version": "1.0.0",
139+
"storages": {
140+
"keyValueStore": {
141+
"actorKeyValueStoreSchemaVersion": 1,
142+
"title": "Key-Value Store Schema",
143+
"collections": {}
144+
}
145+
}
146+
}
147+
```
148+
149+
### Separate configuration files
150+
151+
```json title=".actor/actor.json"
152+
{
153+
"actorSpecification": 1,
154+
"name": "this-is-book-library-scraper",
155+
"title": "Book Library scraper",
156+
"version": "1.0.0",
157+
"storages": {
158+
"keyValueStore": "./key_value_store_schema.json"
159+
}
160+
}
161+
```
162+
163+
```json title=".actor/key_value_store_schema.json"
164+
{
165+
"actorKeyValueStoreSchemaVersion": 1,
166+
"title": "Key-Value Store Schema",
167+
"collections": {}
168+
}
169+
```
170+
171+
Both of these methods are valid so choose one that suits your needs best.
172+
173+
## Key-value store schema structure definitions
174+
175+
The key-value store schema structure defines the various components and properties that govern the organization and representation of the output data produced by an Actor.
176+
It specifies the structure of the data, the transformations to be applied, and the visual display configurations for the Output tab UI.
177+
178+
### Key-value store schema object definition
179+
180+
| Property | Type | Required | Description |
181+
|-----------------------------------|-------------------------------|----------|-----------------------------------------------------------------------------------------------------------------|
182+
| `actorKeyValueStoreSchemaVersion` | integer | true | Specifies the version of key-value store schema structure document. <br/>Currently only version 1 is available. |
183+
| `title` | string | true | - |
184+
| `description` | string | false | - |
185+
| `collections` | Object | true | An object where each key is a collection ID and its value is a collection definition object. |
186+
187+
### Collection object definition
188+
189+
| Property | Type | Required | Description |
190+
|---------------------------|--------------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------|
191+
| `title` | string | true | The title displayed in the UI in the Output tab <br/>and returned in the API. |
192+
| `description` | string | false | A description of the collection that appears in tooltips in the UI and in API responses. |
193+
| `key` | string | conditional* | Defines a single specific key that will be part of this collection. |
194+
| `keyPrefix` | string | conditional* | Defines a prefix for keys that should be included in this collection. |
195+
| `contentTypes` | string array | false | Allowed content types for records in this collection. Used for validation when storing data. |
196+
| `jsonSchema` | object | false | For collections with content type `application/json`, you can define a JSON schema to validate structure. <br/>Uses JsonSchema Draft 07 format. |
197+
198+
\* Either `key` or `keyPrefix` must be specified for each collection, but not both.

0 commit comments

Comments
 (0)