Skip to content

Commit 9a2464b

Browse files
committed
Add DynamoDB store implementation and unit tests
- Introduced exceptions for DynamoDB store operations. - Created unit tests for DynamoDBStore covering initialization, setup, operations, filtering, and batch processing. - Added a Jupyter notebook demonstrating the usage of the DynamoDB store, including setup, data storage, retrieval, searching, filtering, TTL usage, and batch operations.
1 parent 08b2314 commit 9a2464b

File tree

9 files changed

+1759
-2
lines changed

9 files changed

+1759
-2
lines changed
Lines changed: 265 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,265 @@
1+
# DynamoDB Store for LangGraph
2+
3+
A DynamoDB-backed store implementation for LangGraph that provides persistent key-value storage with hierarchical namespaces.
4+
5+
## Features
6+
7+
-**Persistent Storage**: Durable storage using AWS DynamoDB
8+
-**Hierarchical Namespaces**: Organize data with multi-level namespaces
9+
-**TTL Support**: Automatic item expiration with configurable time-to-live
10+
-**Filtering**: Basic filtering capabilities for search operations
11+
-**Batch Operations**: Efficient batch processing of multiple operations
12+
-**Cost-Effective**: Pay-per-request billing for unpredictable workloads
13+
14+
## Installation
15+
16+
```bash
17+
pip install langgraph-checkpoint-aws
18+
```
19+
20+
## Quick Start
21+
22+
```python
23+
from langgraph_checkpoint_aws import DynamoDBStore
24+
25+
# Create a store instance
26+
store = DynamoDBStore(table_name="my-store-table")
27+
28+
# Setup the table (creates it if it doesn't exist)
29+
store.setup()
30+
31+
# Store and retrieve data
32+
store.put(("users", "123"), "prefs", {"theme": "dark"})
33+
item = store.get(("users", "123"), "prefs")
34+
print(item.value) # {"theme": "dark"}
35+
```
36+
37+
## Basic Usage
38+
39+
### Storing Documents
40+
41+
```python
42+
# Store a document with hierarchical namespace
43+
store.put(
44+
("documents", "user123"),
45+
"report_1",
46+
{
47+
"text": "Machine learning report on customer behavior analysis...",
48+
"tags": ["ml", "analytics", "report"],
49+
"author": "data_scientist"
50+
}
51+
)
52+
```
53+
54+
### Retrieving Documents
55+
56+
```python
57+
# Get a specific document
58+
item = store.get(("documents", "user123"), "report_1")
59+
print(f"Text: {item.value['text']}")
60+
print(f"Created: {item.created_at}")
61+
print(f"Updated: {item.updated_at}")
62+
```
63+
64+
### Searching
65+
66+
```python
67+
# Search all documents in a namespace
68+
results = store.search(("documents", "user123"))
69+
70+
# Search with filter
71+
results = store.search(
72+
("documents", "user123"),
73+
filter={"author": "data_scientist"}
74+
)
75+
```
76+
77+
### Deleting Items
78+
79+
```python
80+
store.delete(("documents", "user123"), "report_1")
81+
```
82+
83+
## Advanced Features
84+
85+
### Time-To-Live (TTL)
86+
87+
Configure automatic item expiration:
88+
89+
```python
90+
store = DynamoDBStore(
91+
table_name="my-store-table",
92+
ttl={
93+
"default_ttl": 60, # 60 minutes default TTL
94+
"refresh_on_read": True, # Refresh TTL on reads
95+
}
96+
)
97+
store.setup()
98+
99+
# Item will expire after 60 minutes
100+
store.put(("temp", "session_123"), "data", {"value": "temporary data"})
101+
102+
# Custom TTL for specific item (30 minutes)
103+
store.put(
104+
("temp", "session_123"),
105+
"short_lived",
106+
{"value": "expires soon"},
107+
ttl=30
108+
)
109+
```
110+
111+
### Listing Namespaces
112+
113+
```python
114+
# List all namespaces
115+
namespaces = store.list_namespaces()
116+
117+
# List with prefix filter
118+
user_namespaces = store.list_namespaces(prefix=("users",))
119+
120+
# Limit depth
121+
shallow_namespaces = store.list_namespaces(max_depth=2)
122+
```
123+
124+
### Batch Operations
125+
126+
```python
127+
from langgraph.store.base import PutOp, GetOp
128+
129+
# Batch put operations
130+
ops = [
131+
PutOp(("batch",), "item1", {"value": 1}, None, None),
132+
PutOp(("batch",), "item2", {"value": 2}, None, None),
133+
PutOp(("batch",), "item3", {"value": 3}, None, None),
134+
]
135+
results = store.batch(ops)
136+
137+
# Batch get operations
138+
get_ops = [
139+
GetOp(("batch",), "item1", False),
140+
GetOp(("batch",), "item2", False),
141+
]
142+
items = store.batch(get_ops)
143+
```
144+
145+
### Context Manager
146+
147+
```python
148+
with DynamoDBStore.from_conn_string("my-store-table") as store:
149+
store.setup()
150+
store.put(("test",), "example", {"data": "value"})
151+
item = store.get(("test",), "example")
152+
```
153+
154+
## Configuration Options
155+
156+
### Constructor Parameters
157+
158+
- `table_name` (str): Name of the DynamoDB table
159+
- `region_name` (str, optional): AWS region name
160+
- `boto3_session` (boto3.Session, optional): Custom boto3 session
161+
- `ttl` (TTLConfig, optional): TTL configuration
162+
- `max_read_capacity_units` (int, optional): Max read capacity (default: 10)
163+
- `max_write_capacity_units` (int, optional): Max write capacity (default: 10)
164+
165+
### TTL Configuration
166+
167+
```python
168+
ttl = {
169+
"default_ttl": 60, # Default TTL in minutes
170+
"refresh_on_read": True, # Refresh TTL when items are read
171+
}
172+
```
173+
174+
## DynamoDB Table Schema
175+
176+
The store uses a single DynamoDB table with the following structure:
177+
178+
- **PK** (Partition Key, String): Namespace joined with ':'
179+
- **SK** (Sort Key, String): Item key
180+
- **value** (Map): The stored dictionary
181+
- **created_at** (String): ISO format timestamp
182+
- **updated_at** (String): ISO format timestamp
183+
- **expires_at** (Number, optional): Unix timestamp for TTL
184+
185+
## AWS Configuration
186+
187+
Ensure you have proper AWS credentials configured through:
188+
189+
- Environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`)
190+
- AWS credentials file (`~/.aws/credentials`)
191+
- IAM role when running on AWS services
192+
193+
Required IAM permissions:
194+
195+
```json
196+
{
197+
"Version": "2012-10-17",
198+
"Statement": [
199+
{
200+
"Effect": "Allow",
201+
"Action": [
202+
"dynamodb:CreateTable",
203+
"dynamodb:DescribeTable",
204+
"dynamodb:PutItem",
205+
"dynamodb:GetItem",
206+
"dynamodb:Query",
207+
"dynamodb:Scan",
208+
"dynamodb:DeleteItem",
209+
"dynamodb:UpdateItem",
210+
"dynamodb:UpdateTimeToLive"
211+
],
212+
"Resource": "arn:aws:dynamodb:*:*:table/your-table-name"
213+
}
214+
]
215+
}
216+
```
217+
218+
## Comparison with Other Stores
219+
220+
### DynamoDB Store vs Valkey Store
221+
222+
| Feature | DynamoDB Store | Valkey Store |
223+
|---------|---------------|--------------|
224+
| Vector Search | ❌ No | ✅ Yes |
225+
| High Performance | ✅ Good | ✅ Excellent |
226+
| TTL Support | ✅ Yes | ✅ Yes |
227+
| Cost | Pay-per-request | Infrastructure cost |
228+
| Best For | Simple storage, managed infra | Vector search, high performance |
229+
230+
Use **DynamoDB Store** when:
231+
- You need a fully managed solution
232+
- You don't require vector search capabilities
233+
- You want pay-per-request pricing
234+
- Your workload is unpredictable
235+
236+
Use **Valkey Store** when:
237+
- You need vector search capabilities
238+
- You require ultra-low latency
239+
- You can manage your own infrastructure
240+
- You have consistent, predictable workloads
241+
242+
## Limitations
243+
244+
- **No Vector Search**: This store does not support semantic/vector search
245+
- **Scan Cost**: Listing namespaces uses DynamoDB Scan which can be expensive
246+
- **Filter Limitations**: Basic filtering only (equality checks)
247+
- **No Transactions**: Operations are not transactional across multiple items
248+
249+
## Examples
250+
251+
See the [example notebook](../../samples/memory/dynamodb_store.ipynb) for comprehensive usage examples.
252+
253+
## Contributing
254+
255+
Contributions are welcome! Please see the main [CONTRIBUTING.md](../../libs/langgraph-checkpoint-aws/CONTRIBUTING.md) for guidelines.
256+
257+
## License
258+
259+
This package is part of the `langgraph-checkpoint-aws` project. See [LICENSE](../../LICENSE) for details.
260+
261+
## Related Resources
262+
263+
- [LangGraph Documentation](https://langchain-ai.github.io/langgraph/)
264+
- [AWS DynamoDB Documentation](https://docs.aws.amazon.com/dynamodb/)
265+
- [BaseStore Interface](https://langchain-ai.github.io/langgraph/reference/store/)

libs/langgraph-checkpoint-aws/langgraph_checkpoint_aws/__init__.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,9 @@
4545
ValkeyConnectionError,
4646
ValkeyStoreError,
4747
)
48+
from langgraph_checkpoint_aws.store.dynamodb import (
49+
DynamoDBStore,
50+
)
4851

4952
valkey_available = True
5053
except ImportError as e:
@@ -102,6 +105,7 @@ def _missing_dependencies_error(*args: Any, **kwargs: Any) -> Any:
102105
"ValkeySaver",
103106
"ValkeyCache",
104107
"DynamoDBSaver",
108+
"DynamoDBStore",
105109
"SDK_USER_AGENT",
106110
"valkey_available",
107111
]

libs/langgraph-checkpoint-aws/langgraph_checkpoint_aws/store/__init__.py

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,9 @@
88
# Conditional imports for optional dependencies
99
try:
1010
from .valkey import AsyncValkeyStore, ValkeyIndexConfig, ValkeyStore
11+
from .dynamodb import DynamoDBStore
1112

12-
__all__ = ["AsyncValkeyStore", "ValkeyStore", "ValkeyIndexConfig"]
13+
__all__ = ["AsyncValkeyStore", "ValkeyStore", "ValkeyIndexConfig", "DynamoDBStore"]
1314
except ImportError as e:
1415
# Store the error for later use
1516
_import_error = e
@@ -26,5 +27,6 @@ def _missing_dependencies_error(*args: Any, **kwargs: Any) -> Any:
2627
AsyncValkeyStore: type[Any] = _missing_dependencies_error # type: ignore[assignment,no-redef]
2728
ValkeyIndexConfig: type[Any] = _missing_dependencies_error # type: ignore[assignment,no-redef]
2829
ValkeyStore: type[Any] = _missing_dependencies_error # type: ignore[assignment,no-redef]
30+
DynamoDBStore: type[Any] = _missing_dependencies_error # type: ignore[assignment,no-redef]
2931

30-
__all__ = ["AsyncValkeyStore", "ValkeyStore", "ValkeyIndexConfig"]
32+
__all__ = ["AsyncValkeyStore", "ValkeyStore", "ValkeyIndexConfig", "DynamoDBStore"]
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
"""DynamoDB store implementation for LangGraph checkpoint AWS."""
2+
3+
from langgraph_checkpoint_aws.store.dynamodb.base import DynamoDBStore
4+
5+
__all__ = ["DynamoDBStore"]

0 commit comments

Comments
 (0)