Working memory promotion creates duplicate long-term memories (bypasses deduplication)

[abrookins]

Summary

When memories are promoted from working memory to long-term storage via set_working_memory, duplicates are created. However, when using create_long_term_memories directly, duplicates are correctly detected and merged.

Root Cause

The promote_working_memory_to_long_term() function calls index_long_term_memories() with deduplicate=False, bypassing hash-based and semantic deduplication.

Working path (correctly deduplicates):

• create_long_term_memories → index_long_term_memories(..., deduplicate=True)
• Runs deduplicate_by_hash() and deduplicate_by_semantic_search()

Broken path (creates duplicates):

• set_working_memory → promote_working_memory_to_long_term() → index_long_term_memories(..., deduplicate=False)
• Only runs deduplicate_by_id() which catches exact ID matches, not content duplicates

Code Location

agent_memory_server/long_term_memory.py, lines 1330 and 1410:

await index_long_term_memories(
    [current_memory],
    redis_client=redis,
    deduplicate=False,  # <-- Should be True
)

The comment says "Already deduplicated by id" but ID-based deduplication only catches overwrites of the same memory ID, not:

• Hash duplicates: Same text, different IDs
• Semantic duplicates: Similar meaning, different wording

Evidence

Logs from create_long_term_memories show deduplication working:

Found existing memory with hash 59a93cb054cd39dfd39973837af711e9cf800f1e64ca114804ce4aa9cf4be11d
Merged new memory with 1 semantic duplicates
Indexed 5 memories with IDs: [...]

Logs from set_working_memory promotion show no deduplication:

Promoted new memory with id 01KEJ3TSM0HGPFV7D9D35PP45X
Promoted new memory with id 01KEJ3TSM0HGPFV7D9D35PP45Y
...
Successfully promoted 13 memories

Redis contains clear duplicates with the same semantic content:

• "Andrew likes apples" (hash: 0b91b...)
• "User likes apples." (hash: 6a640...)
• "User likes apples." (hash: f6ea0...)

Suggested Fix

Change deduplicate=False to deduplicate=True in the two index_long_term_memories() calls within promote_working_memory_to_long_term().