Feature Request: Add L0/L1/L2 Hierarchical Memory Layer inspired by ai-agent-memory

[Brisbanehuang]

Summary

I'd like to propose adding a three-tier hierarchical memory layer (L0/L1/L2) to memory-lancedb-pro, inspired by the architecture in jzOcb/ai-agent-memory.

While memory-lancedb-pro excels at hybrid retrieval (Vector + BM25) and cross-encoder reranking for large-scale semantic search, the L0/L1/L2 hierarchy from ai-agent-memory offers a complementary approach that could significantly reduce token consumption and improve retrieval efficiency.

---

Background: What is L0/L1/L2?

The hierarchical memory system from ai-agent-memory organizes memories into three layers:

Layer	Location	Content	When to Read
L0	.abstract files	Directory overview (one-sentence index)	Always first
L1	insights/, lessons/	Distilled patterns and structured summaries	On demand
L2	YYYY-MM-DD.md	Full daily logs and raw narratives	Deep dive only

Key benefit: 90% of queries need only L0 + L1, dramatically reducing token usage while maintaining context quality.

This design is inspired by OpenViking (ByteDance) and aligns with the "reflection" mechanism from Stanford Generative Agents.

---

Why This Complements memory-lancedb-pro

memory-lancedb-pro already has excellent features:

• ✅ Hybrid retrieval (Vector + BM25)
• ✅ Cross-encoder reranking
• ✅ Smart extraction with 6-category classification
• ✅ Weibull decay + 3-tier promotion (Peripheral ⟷ Working ⟷ Core)

However, the current system treats all memories at the same granularity level. Adding L0/L1/L2 would:

1. Reduce token consumption: Most queries can be answered with L0 (index) + L1 (summary) without loading full L2 (narrative)
2. Improve retrieval precision: L0 acts as a fast filter before expensive vector search
3. Enable progressive disclosure: Start with L0 overview → drill down to L1 insights → access L2 details only when needed
4. Complement existing features: L0/L1/L2 can work alongside the existing 6-category system (profile, preferences, entities, events, cases, patterns)

---

Proposed Implementation

Option A: Metadata-Based Layers (Minimal Change)

Add a layer field to the existing memory schema:

interface Memory {
  // ... existing fields ...
  layer: 'L0' | 'L1' | 'L2';  // New field
  l0_index?: string;          // One-sentence summary for L0
  l1_summary?: string;        // Structured summary for L1
  l2_narrative?: string;      // Full content for L2
}

Retrieval flow:

1. Query L0 entries first (fast, low token cost)
2. If L0 matches, retrieve corresponding L1 summaries
3. Only fetch L2 narratives when explicitly needed

Option B: Separate Tables (Clean Separation)

Create three LanceDB tables:

• memories_l0 — Index layer (one-sentence per memory)
• memories_l1 — Summary layer (structured insights)
• memories_l2 — Narrative layer (full content)

Link them via memory_id foreign key.

Option C: Hybrid Approach

• Keep existing memories table for L2 (full content)
• Add memories_index table for L0/L1 (lightweight summaries)
• Use existing smart extraction to automatically generate L0/L1 from L2

---

Integration with Existing Features

1. Smart Extraction Enhancement

Extend the existing LLM-powered extraction to generate all three layers:

// Current: 6-category extraction
{ category: 'profile', content: '...' }

// Proposed: 6-category + 3-layer extraction
{
  category: 'profile',
  l0_index: 'User prefers dark mode',
  l1_summary: 'User explicitly requested dark mode for all interfaces',
  l2_narrative: 'During setup, user mentioned they work late nights and find dark mode easier on the eyes...'
}

2. Adaptive Retrieval

Modify src/adaptive-retrieval.ts to:

• Start with L0 for simple queries
• Escalate to L1 for moderate complexity
• Access L2 only for deep context needs

3. Lifecycle Management

Apply existing Weibull decay + tier promotion to all layers:

• L0 entries decay faster (lightweight, easy to regenerate)
• L1 entries have medium decay (valuable insights)
• L2 entries decay slower (full context preservation)

---

Benefits

1. Token Efficiency: Reduce context window usage by 70-90% for typical queries
2. Faster Retrieval: L0 index enables quick filtering before expensive vector search
3. Better UX: Progressive disclosure matches human information-seeking behavior
4. Backward Compatible: Can be implemented as an optional feature flag
5. Synergy with Existing Features: Complements rather than replaces current architecture

---

References

• jzOcb/ai-agent-memory — Original L0/L1/L2 implementation
• OpenViking (ByteDance) — L0/L1/L2 hierarchy inspiration
• Stanford Generative Agents — Reflection mechanism

---

Questions for Discussion

1. Which implementation approach (A/B/C) would fit best with the current architecture?
2. Should L0/L1/L2 be opt-in (feature flag) or always-on?
3. How should the existing 6-category system interact with the 3-layer hierarchy?
4. Should we add a CLI command like openclaw memory-pro compact to generate L0/L1 from existing L2 memories?

I'm happy to contribute to the implementation if this aligns with the project's roadmap. Thank you for building such a powerful memory system!

[AliceLJY]

Thanks for the detailed and well-structured proposal! Really appreciate the time you put into researching the references and laying out the implementation options.

Just wanted to share some context on where things stand today:

memory-lancedb-pro already has a 3-tier system (Peripheral ⟷ Working ⟷ Core) with Weibull decay and promotion based on access frequency and importance. The proposed L0/L1/L2 takes a different angle — layering by granularity (index/summary/full) rather than importance — which is an interesting idea. That said, combining both hierarchies would add quite a bit of architectural complexity, so we'd want to be thoughtful about the tradeoff.

On the token efficiency side, the current retrieval pipeline is already search-based (hybrid vector + BM25 → rerank → top-K), so it's not loading all memories into context. The token cost is mainly in embedding and reranking rather than in reading raw content, which means an additional index layer might not yield as much savings as expected in practice.

That said, we definitely don't want to dismiss the idea — it could be valuable in certain scenarios. To help us evaluate, could you share what specific pain points you're running into? For example, are you hitting token limits, seeing retrieval quality issues, or experiencing latency problems? A concrete use case would really help us figure out whether L0/L1/L2 is the right approach or if there's a simpler way to address it within the existing architecture.

Thanks again for the thoughtful suggestion! 🙏