Skip to content

Commit 738ad76

Browse files
committed
docs: clarify dual-memory architecture (fixes #344)
Add three improvements addressing Issue #344: 1. README: add Dual-Memory Architecture section explaining Plugin Memory (LanceDB) vs Markdown Memory distinction 2. index.ts: log dual-memory warning on plugin startup 3. cli.ts: add import-markdown command to migrate existing Markdown memories into the plugin store for semantic recall Refs: #344
1 parent 7fe2ae0 commit 738ad76

File tree

3 files changed

+159
-22
lines changed

3 files changed

+159
-22
lines changed

README.md

Lines changed: 25 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -9,19 +9,10 @@
99
A LanceDB-backed OpenClaw memory plugin that stores preferences, decisions, and project context, then auto-recalls them in future sessions.
1010

1111
[![OpenClaw Plugin](https://img.shields.io/badge/OpenClaw-Plugin-blue)](https://github.com/openclaw/openclaw)
12-
[![OpenClaw 2026.3+](https://img.shields.io/badge/OpenClaw-2026.3%2B-brightgreen)](https://github.com/openclaw/openclaw)
1312
[![npm version](https://img.shields.io/npm/v/memory-lancedb-pro)](https://www.npmjs.com/package/memory-lancedb-pro)
1413
[![LanceDB](https://img.shields.io/badge/LanceDB-Vectorstore-orange)](https://lancedb.com)
1514
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
1615

17-
<h2>⚡ <a href="https://github.com/CortexReach/memory-lancedb-pro/releases/tag/v1.1.0-beta.10">v1.1.0-beta.10 — OpenClaw 2026.3+ Hook Adaptation</a></h2>
18-
19-
<p>
20-
✅ Fully adapted for OpenClaw 2026.3+ new plugin architecture<br>
21-
🔄 Uses <code>before_prompt_build</code> hooks (replacing deprecated <code>before_agent_start</code>)<br>
22-
🩺 Run <code>openclaw doctor --fix</code> after upgrading
23-
</p>
24-
2516
[English](README.md) | [简体中文](README_CN.md) | [繁體中文](README_TW.md) | [日本語](README_JA.md) | [한국어](README_KO.md) | [Français](README_FR.md) | [Español](README_ES.md) | [Deutsch](README_DE.md) | [Italiano](README_IT.md) | [Русский](README_RU.md) | [Português (Brasil)](README_PT-BR.md)
2617

2718
</div>
@@ -129,6 +120,31 @@ Add to your `openclaw.json`:
129120
- `extractMinMessages: 2` → extraction triggers in normal two-turn chats
130121
- `sessionMemory.enabled: false` → avoids polluting retrieval with session summaries on day one
131122

123+
---
124+
125+
## ⚠️ Dual-Memory Architecture (Important)
126+
127+
When `memory-lancedb-pro` is active, your system has **two independent memory layers** that do **not** auto-sync:
128+
129+
| Memory Layer | Storage | What it's for | Recallable? |
130+
|---|---|---|---|
131+
| **Plugin Memory** | LanceDB (vector store) | Semantic recall via `memory_recall` / auto-recall | ✅ Yes |
132+
| **Markdown Memory** | `MEMORY.md`, `memory/YYYY-MM-DD.md` | Startup context, human-readable journal | ❌ Not auto-recalled |
133+
134+
**Key principle:**
135+
> A fact written into `memory/YYYY-MM-DD.md` is visible in startup context, but `memory_recall` **will not find it** unless it was also written via `memory_store` (or auto-captured by the plugin).
136+
137+
**What this means for you:**
138+
- Need semantic recall? → Use `memory_store` or let auto-capture do it
139+
- `memory/YYYY-MM-DD.md` → treat as a **daily journal / log**, not a recall source
140+
- `MEMORY.md` → curated human-readable reference, not a recall source
141+
- Plugin memory → **primary recall source** for `memory_recall` and auto-recall
142+
143+
**If you want your Markdown memories to be recallable**, use the import command:
144+
```bash
145+
npx memory-lancedb-pro memory-pro import-markdown
146+
```
147+
132148
Validate & restart:
133149

134150
```bash
@@ -620,19 +636,6 @@ Sometimes the model may echo the injected `<relevant-memories>` block.
620636
621637
</details>
622638

623-
<details>
624-
<summary><strong>Auto-recall timeout tuning</strong></summary>
625-
626-
Auto-recall has a configurable timeout (default 5s) to prevent stalling agent startup. If you're behind a proxy or using a high-latency embedding API, increase it:
627-
628-
```json
629-
{ "plugins": { "entries": { "memory-lancedb-pro": { "config": { "autoRecallTimeoutMs": 8000 } } } } }
630-
```
631-
632-
If auto-recall consistently times out, check your embedding API latency first. The timeout only affects the automatic injection path — manual `memory_recall` tool calls are not affected.
633-
634-
</details>
635-
636639
<details>
637640
<summary><strong>Session Memory</strong></summary>
638641

cli.ts

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1036,6 +1036,131 @@ export function registerMemoryCLI(program: Command, context: CLIContext): void {
10361036
}
10371037
});
10381038

1039+
/**
1040+
* import-markdown: Import memories from Markdown memory files into the plugin store.
1041+
* Targets MEMORY.md and memory/YYYY-MM-DD.md files found in OpenClaw workspaces.
1042+
*/
1043+
memory
1044+
.command("import-markdown [workspace-glob]")
1045+
.description("Import memories from Markdown files (MEMORY.md, memory/YYYY-MM-DD.md) into the plugin store")
1046+
.option("--dry-run", "Show what would be imported without importing")
1047+
.option("--scope <scope>", "Import into specific scope (default: global)")
1048+
.option(
1049+
"--openclaw-home <path>",
1050+
"OpenClaw home directory (default: ~/.openclaw)",
1051+
)
1052+
.action(async (workspaceGlob, options) => {
1053+
const openclawHome = options.openclawHome
1054+
? path.resolve(options.openclawHome)
1055+
: path.join(homedir(), ".openclaw");
1056+
1057+
const workspaceDir = path.join(openclawHome, "workspace");
1058+
let imported = 0;
1059+
let skipped = 0;
1060+
let foundFiles = 0;
1061+
1062+
if (!context.embedder) {
1063+
console.error(
1064+
"import-markdown requires an embedder. Use via plugin CLI or ensure embedder is configured.",
1065+
);
1066+
process.exit(1);
1067+
}
1068+
1069+
// Scan workspace directories
1070+
let workspaceEntries: string[];
1071+
try {
1072+
const fsPromises = await import("node:fs/promises");
1073+
workspaceEntries = await fsPromises.readdir(workspaceDir, { withFileTypes: true });
1074+
} catch {
1075+
console.error(`Failed to read workspace directory: ${workspaceDir}`);
1076+
process.exit(1);
1077+
}
1078+
1079+
// Collect all markdown files to scan
1080+
const mdFiles: Array<{ filePath: string; scope: string }> = [];
1081+
1082+
for (const entry of workspaceEntries) {
1083+
if (!entry.isDirectory()) continue;
1084+
if (workspaceGlob && !entry.name.includes(workspaceGlob)) continue;
1085+
1086+
const workspacePath = path.join(workspaceDir, entry.name);
1087+
1088+
// MEMORY.md
1089+
const memoryMd = path.join(workspacePath, "MEMORY.md");
1090+
try {
1091+
const { stat } = await import("node:fs/promises");
1092+
await stat(memoryMd);
1093+
mdFiles.push({ filePath: memoryMd, scope: entry.name });
1094+
} catch { /* not found */ }
1095+
1096+
// memory/ directory
1097+
const memoryDir = path.join(workspacePath, "memory");
1098+
try {
1099+
const { stat } = await import("node:fs/promises");
1100+
const stats = await stat(memoryDir);
1101+
if (stats.isDirectory()) {
1102+
const { readdir } = await import("node:fs/promises");
1103+
const files = await readdir(memoryDir);
1104+
for (const f of files) {
1105+
if (f.endsWith(".md") && /^\d{4}-\d{2}-\d{2}/.test(f)) {
1106+
mdFiles.push({ filePath: path.join(memoryDir, f), scope: entry.name });
1107+
}
1108+
}
1109+
}
1110+
} catch { /* not found */ }
1111+
}
1112+
1113+
if (mdFiles.length === 0) {
1114+
console.log("No Markdown memory files found.");
1115+
return;
1116+
}
1117+
1118+
const targetScope = options.scope || "global";
1119+
1120+
// Parse each file for memory entries (lines starting with "- ")
1121+
for (const { filePath, scope } of mdFiles) {
1122+
foundFiles++;
1123+
const { readFile } = await import("node:fs/promises");
1124+
const content = await readFile(filePath, "utf-8");
1125+
const lines = content.split("\n");
1126+
1127+
for (const line of lines) {
1128+
// Skip non-memory lines
1129+
if (!line.startsWith("- ")) continue;
1130+
const text = line.slice(2).trim();
1131+
if (text.length < 5) { skipped++; continue; }
1132+
1133+
if (options.dryRun) {
1134+
console.log(` [dry-run] would import: ${text.slice(0, 80)}...`);
1135+
imported++;
1136+
continue;
1137+
}
1138+
1139+
try {
1140+
const vector = await context.embedder!.embedQuery(text);
1141+
await context.store.store({
1142+
text,
1143+
vector,
1144+
importance: 0.7,
1145+
category: "other",
1146+
scope: targetScope,
1147+
metadata: { importedFrom: filePath, sourceScope: scope },
1148+
});
1149+
imported++;
1150+
} catch (err) {
1151+
console.warn(` Failed to import: ${text.slice(0, 60)}... — ${err}`);
1152+
skipped++;
1153+
}
1154+
}
1155+
}
1156+
1157+
if (options.dryRun) {
1158+
console.log(`\nDRY RUN — found ${foundFiles} files, ${imported} entries would be imported, ${skipped} skipped`);
1159+
} else {
1160+
console.log(`\nImport complete: ${imported} imported, ${skipped} skipped (scanned ${foundFiles} files)`);
1161+
}
1162+
});
1163+
10391164
// Re-embed an existing LanceDB into the current target DB (A/B testing)
10401165
memory
10411166
.command("reembed")

index.ts

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1993,6 +1993,15 @@ const memoryLanceDBProPlugin = {
19931993
);
19941994
logReg(`memory-lancedb-pro: diagnostic build tag loaded (${DIAG_BUILD_TAG})`);
19951995

1996+
// Dual-memory model warning: help users understand the two-layer architecture
1997+
// Runs synchronously and logs warnings; does NOT block gateway startup.
1998+
api.logger.info(
1999+
`[memory-lancedb-pro] memory_recall queries the plugin store (LanceDB), not MEMORY.md.\n` +
2000+
` - Plugin memory (LanceDB) = primary recall source for semantic search\n` +
2001+
` - MEMORY.md / memory/YYYY-MM-DD.md = startup context / journal only\n` +
2002+
` - Use memory_store or auto-capture for recallable memories.\n`
2003+
);
2004+
19962005
api.on("message_received", (event: any, ctx: any) => {
19972006
const conversationKey = buildAutoCaptureConversationKeyFromIngress(
19982007
ctx.channelId,

0 commit comments

Comments
 (0)