mem0-hybrid-search

Chinese memory hybrid retrieval booster: CJK bigram chunking + weighted RRF fusion + MMR dedup + recency decay.

Zero-dependency, pure functions. Plug it on top of any vector-only memory store (mem0 / opencode-mem0 / SQLite + vectors) to fix the classic Chinese retrieval blind spots — names, IDs, short tokens that pure vector search simply can't find.

中文文档 · npm · MIT

Why

Pure vector search has three blind spots with Chinese text:

Query	Pure vector	This package
林小满 (a person's name)	❌ missed	✅ bigram splits into 林小/小满, hits
4747 (pure digits / ID)	❌ missed	✅ FTS channel tokenizes digits
库存对不上 (colloquial)	⚠️ semantic drift	✅ FTS keywords + bigram double-precision channels

Architecture

① FTS5 trigram keyword search → exact match
② Vector cosine similarity    → semantic match (optional; auto-degrades when no vectors)
③ CJK bigram chunking        → Chinese blind-spot rescue (names/IDs/short tokens)
        ↓
Weighted RRF fusion (precision channels ×2, fixes equal-weight dilution)
        ↓
MMR dedup + recency decay + threshold filter + normalization

Install

npm install mem0-hybrid-search

Usage

const { hybridFuse, extractCjkBigrams } = require("mem0-hybrid-search");

// 1. Candidate lists from your own retrieval (FTS / vector / bigram channels)
const candidates = [
  [{ id: "a", score: 0.9 }, { id: "b", score: 0.7 }],  // FTS channel
  [{ id: "b", score: 0.8 }, { id: "c", score: 0.6 }],  // vector channel
  [{ id: "a", score: 0.5 }],                            // bigram channel
];

// 2. Per-candidate metadata (used by recency decay & MMR)
const meta = {
  a: { createdAt: Date.now() - 86400000,     vector: [0.1, 0.2] },
  b: { createdAt: Date.now() - 86400000 * 7, vector: [0.3, 0.1] },
  c: { createdAt: Date.now(),                vector: [0.2, 0.2] },
};

// 3. Fuse & rank
const hits = hybridFuse(candidates, { maxResults: 3 }, meta);

// 4. Or use CJK bigram extraction standalone
const bigrams = extractCjkBigrams("林小满"); // → ["林小", "小满"]

API

Function	Description
hybridFuse(candidates, opts, meta)	Full pipeline: RRF → MMR → recency → threshold → normalize
rrfFuse(lists, k, weights)	Weighted Reciprocal Rank Fusion
mmrRerank(candidates, getEmbedding, lambda, topK)	MMR diversity reranking
applyRecencyDecay(candidates, halfLifeDays)	Exponential time decay (14-day half-life)
extractCjkBigrams(query)	CJK 2-char sliding window (incl. CJK ext. regions)

Relationship to mem0 official

• RRF / FTS5 / vector fusion: industry-standard approaches — the official mem0 ships hybrid search too. This package shares that fusion framework.
• What this package adds: ① a CJK bigram channel (fixes Chinese name/ID blind spots) ② 2× weighting for precision channels (fixes the equal-weight dilution bug where 25 noisy vector candidates drown 2 exact hits). These are pure Chinese-scenario supplements, independent of any memory backend.

MCP Integration

The same engine is exposed as a standard MCP (Model Context Protocol) memory server — one codebase, any MCP-capable client (opencode / DeepSeek Harness / Claude Desktop / Cursor …):

{
  "mcp": {
    "mem0-hybrid-search": {
      "type": "local",
      "command": ["node", "/path/to/mem0-hybrid-search/mcp-server.js"],
      "environment": {}
    }
  }
}

Tools: memory_search (hybrid retrieval), memory_sync (incremental index sync), memory_stats (index status). Data stays local — index DB and mem0 shards are only read/written on your machine.

The full sidecar (engine + MCP server + opencode plugin) lives at ~/.config/opencode/plugins/mem0-hybrid-search/. This npm package ships the pure algorithm layer.

Test

npm test

12 assertions covering bigram extraction, weighted-RRF-vs-equal-RRF, full pipeline, recency decay, and MMR.

License

MIT