Skip to content

Commit aca481d

Browse files
DimaJoytiDimaJoyti
committed
Distributed memory
1 parent dbf4231 commit aca481d

File tree

9 files changed

+2213
-198
lines changed

9 files changed

+2213
-198
lines changed

‎docs/api.md‎

Lines changed: 44 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -157,11 +157,7 @@ X-API-Key: key1
157157
"agent_id": "unique-agent-id",
158158
"agent_mode": "basic",
159159
"status": "available",
160-
"capabilities": [
161-
"chat",
162-
"web_search",
163-
"web_browsing"
164-
],
160+
"capabilities": ["chat", "web_search", "web_browsing"],
165161
"created_at": "2023-01-01T00:00:00Z",
166162
"metadata": {
167163
"description": "Basic Agent"
@@ -273,25 +269,67 @@ When rate limiting is enabled, the API will return a 429 Too Many Requests respo
273269

274270
The API can be configured using environment variables:
275271

272+
### API Settings
273+
276274
- `API_TITLE`: API title
277275
- `API_DESCRIPTION`: API description
278276
- `API_VERSION`: API version
279277
- `API_OPENAPI_URL`: OpenAPI URL
280278
- `API_DOCS_URL`: Swagger UI URL
281279
- `API_REDOC_URL`: ReDoc URL
280+
281+
### Server Settings
282+
282283
- `API_HOST`: Host to bind the server to
283284
- `API_PORT`: Port to bind the server to
284285
- `API_DEBUG`: Enable debug mode
285286
- `API_RELOAD`: Enable auto-reload on code changes
287+
288+
### Security Settings
289+
286290
- `API_ENABLE_AUTH`: Enable authentication
287291
- `API_KEY_HEADER`: API key header
288292
- `API_KEYS`: Comma-separated list of API keys
293+
294+
### CORS Settings
295+
289296
- `API_ALLOW_ORIGINS`: Comma-separated list of allowed origins for CORS
290297
- `API_ALLOW_METHODS`: Comma-separated list of allowed methods for CORS
291298
- `API_ALLOW_HEADERS`: Comma-separated list of allowed headers for CORS
299+
300+
### Rate Limiting
301+
292302
- `API_ENABLE_RATE_LIMITING`: Enable rate limiting
293303
- `API_RATE_LIMIT_PER_MINUTE`: Rate limit per minute
304+
305+
### Logging
306+
294307
- `API_LOG_LEVEL`: Log level
308+
309+
### Agent Settings
310+
295311
- `API_DEFAULT_AGENT_MODE`: Default agent mode
296-
- `API_MEMORY_BACKEND`: Memory backend
297312
- `API_ENABLE_ALL_TOOLS`: Enable all tools
313+
314+
### Memory Settings
315+
316+
- `API_MEMORY_BACKEND`: Memory backend (sqlite, file, redis, mongodb)
317+
318+
### Redis Settings
319+
320+
- `API_REDIS_HOST`: Redis host
321+
- `API_REDIS_PORT`: Redis port
322+
- `API_REDIS_DB`: Redis database
323+
- `API_REDIS_PASSWORD`: Redis password
324+
- `API_REDIS_PREFIX`: Redis key prefix
325+
326+
### MongoDB Settings
327+
328+
- `API_MONGODB_URI`: MongoDB URI
329+
- `API_MONGODB_DB`: MongoDB database
330+
331+
### Distributed Settings
332+
333+
- `API_ENABLE_DISTRIBUTED`: Enable distributed mode
334+
- `API_DISTRIBUTED_BACKEND`: Distributed backend (redis, mongodb)
335+
- `API_SESSION_STORE`: Session store (redis, mongodb, memory)

‎docs/distributed.md‎

Lines changed: 232 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,232 @@
1+
# Distributed Configuration
2+
3+
This document describes how to configure the API for distributed operation.
4+
5+
## Overview
6+
7+
The API supports distributed operation using Redis or MongoDB as a backend for:
8+
9+
- Session storage
10+
- Memory persistence
11+
- Tool usage logging
12+
- Conversation history
13+
14+
This allows multiple API instances to share state and work together in a horizontally scaled environment.
15+
16+
## Redis Configuration
17+
18+
### Environment Variables
19+
20+
To enable distributed operation with Redis, set the following environment variables:
21+
22+
```bash
23+
# Enable distributed mode
24+
API_ENABLE_DISTRIBUTED=true
25+
API_DISTRIBUTED_BACKEND=redis
26+
API_SESSION_STORE=redis
27+
28+
# Redis connection settings
29+
API_REDIS_HOST=localhost
30+
API_REDIS_PORT=6379
31+
API_REDIS_DB=0
32+
API_REDIS_PASSWORD=your_password
33+
API_REDIS_PREFIX=datamcp:
34+
35+
# Memory backend
36+
API_MEMORY_BACKEND=redis
37+
```
38+
39+
### Docker Compose Example
40+
41+
Here's an example Docker Compose configuration for running the API with Redis:
42+
43+
```yaml
44+
version: '3'
45+
46+
services:
47+
api1:
48+
build: .
49+
ports:
50+
- "8000:8000"
51+
environment:
52+
- API_ENABLE_DISTRIBUTED=true
53+
- API_DISTRIBUTED_BACKEND=redis
54+
- API_SESSION_STORE=redis
55+
- API_REDIS_HOST=redis
56+
- API_REDIS_PORT=6379
57+
- API_REDIS_DB=0
58+
- API_REDIS_PASSWORD=your_password
59+
- API_REDIS_PREFIX=datamcp:
60+
- API_MEMORY_BACKEND=redis
61+
depends_on:
62+
- redis
63+
64+
api2:
65+
build: .
66+
ports:
67+
- "8001:8000"
68+
environment:
69+
- API_ENABLE_DISTRIBUTED=true
70+
- API_DISTRIBUTED_BACKEND=redis
71+
- API_SESSION_STORE=redis
72+
- API_REDIS_HOST=redis
73+
- API_REDIS_PORT=6379
74+
- API_REDIS_DB=0
75+
- API_REDIS_PASSWORD=your_password
76+
- API_REDIS_PREFIX=datamcp:
77+
- API_MEMORY_BACKEND=redis
78+
depends_on:
79+
- redis
80+
81+
redis:
82+
image: redis:7
83+
ports:
84+
- "6379:6379"
85+
command: redis-server --requirepass your_password
86+
volumes:
87+
- redis-data:/data
88+
89+
volumes:
90+
redis-data:
91+
```
92+
93+
## MongoDB Configuration
94+
95+
### Environment Variables
96+
97+
To enable distributed operation with MongoDB, set the following environment variables:
98+
99+
```bash
100+
# Enable distributed mode
101+
API_ENABLE_DISTRIBUTED=true
102+
API_DISTRIBUTED_BACKEND=mongodb
103+
API_SESSION_STORE=mongodb
104+
105+
# MongoDB connection settings
106+
API_MONGODB_URI=mongodb://localhost:27017
107+
API_MONGODB_DB=datamcp
108+
109+
# Memory backend
110+
API_MEMORY_BACKEND=mongodb
111+
```
112+
113+
### Docker Compose Example
114+
115+
Here's an example Docker Compose configuration for running the API with MongoDB:
116+
117+
```yaml
118+
version: '3'
119+
120+
services:
121+
api1:
122+
build: .
123+
ports:
124+
- "8000:8000"
125+
environment:
126+
- API_ENABLE_DISTRIBUTED=true
127+
- API_DISTRIBUTED_BACKEND=mongodb
128+
- API_SESSION_STORE=mongodb
129+
- API_MONGODB_URI=mongodb://mongodb:27017
130+
- API_MONGODB_DB=datamcp
131+
- API_MEMORY_BACKEND=mongodb
132+
depends_on:
133+
- mongodb
134+
135+
api2:
136+
build: .
137+
ports:
138+
- "8001:8000"
139+
environment:
140+
- API_ENABLE_DISTRIBUTED=true
141+
- API_DISTRIBUTED_BACKEND=mongodb
142+
- API_SESSION_STORE=mongodb
143+
- API_MONGODB_URI=mongodb://mongodb:27017
144+
- API_MONGODB_DB=datamcp
145+
- API_MEMORY_BACKEND=mongodb
146+
depends_on:
147+
- mongodb
148+
149+
mongodb:
150+
image: mongo:6
151+
ports:
152+
- "27017:27017"
153+
volumes:
154+
- mongodb-data:/data/db
155+
156+
volumes:
157+
mongodb-data:
158+
```
159+
160+
## Load Balancing
161+
162+
To distribute traffic across multiple API instances, you can use a load balancer like Nginx, HAProxy, or a cloud load balancer.
163+
164+
### Nginx Example
165+
166+
Here's an example Nginx configuration for load balancing:
167+
168+
```nginx
169+
upstream datamcp_api {
170+
server api1:8000;
171+
server api2:8000;
172+
# Add more servers as needed
173+
}
174+
175+
server {
176+
listen 80;
177+
server_name api.example.com;
178+
179+
location / {
180+
proxy_pass http://datamcp_api;
181+
proxy_set_header Host $host;
182+
proxy_set_header X-Real-IP $remote_addr;
183+
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
184+
proxy_set_header X-Forwarded-Proto $scheme;
185+
}
186+
}
187+
```
188+
189+
## Session Persistence
190+
191+
When using a load balancer, you should ensure that requests from the same client are routed to the same API instance (session persistence or sticky sessions). This is important for WebSocket connections and streaming responses.
192+
193+
### Nginx Example with Sticky Sessions
194+
195+
```nginx
196+
upstream datamcp_api {
197+
ip_hash; # This ensures requests from the same IP go to the same server
198+
server api1:8000;
199+
server api2:8000;
200+
# Add more servers as needed
201+
}
202+
203+
server {
204+
listen 80;
205+
server_name api.example.com;
206+
207+
location / {
208+
proxy_pass http://datamcp_api;
209+
proxy_set_header Host $host;
210+
proxy_set_header X-Real-IP $remote_addr;
211+
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
212+
proxy_set_header X-Forwarded-Proto $scheme;
213+
214+
# WebSocket support
215+
proxy_http_version 1.1;
216+
proxy_set_header Upgrade $http_upgrade;
217+
proxy_set_header Connection "upgrade";
218+
}
219+
}
220+
```
221+
222+
## Monitoring
223+
224+
When running in a distributed environment, it's important to monitor the health and performance of your API instances and backend services.
225+
226+
You can use tools like Prometheus, Grafana, and ELK stack for monitoring and logging.
227+
228+
## Scaling
229+
230+
To scale the API horizontally, you can add more API instances to your deployment. The shared state in Redis or MongoDB ensures that all instances work together seamlessly.
231+
232+
You can also scale the Redis or MongoDB backends for better performance and reliability.

0 commit comments

Comments
 (0)