Updating saved memories via chat (tool call) is buggy sporadically

[jhmonroe]

What happened?

1. THANK YOU! Loving testing the new memory feature.

2. I'm testing manually adding memories as well as having the chat update memories and add new memories in the sidebar. It seems to work sporadically... sometimes the new/updated memories will appear in right sidebar. Other times the chat will remember it but it only seems to exist in the database for the chat but does not appear in the right sidebar.

Is there an exact wording we should follow to get the chat to update the memories list or do memory-related things exactly on the first try?
A prompting guide for LibreChat's unique lingo (ie: tool call names in this case) might be helpful!

I don't see mention of it here: https://www.librechat.ai/docs/features/memory
or here: https://www.librechat.ai/docs/configuration/librechat_yaml/object_structure/memory
(other than some of the lingo in valid key examples)

Steps to Reproduce

My process:

• I created 2 memories manually in right sidebar as a test (my name and my sister's name)
• Then I quizzed the chat on my memories (it passed) and instructed the chat to remember a new item about my father's name.
• It did so and when I asked for all the current memories it ran the tool call that shows up in the chat as "Updated saved memory" with a little icon and the new memory (of my father's name) appeared in the list of my previously manually created memories and refreshed the sidebar automatically like magic!
• Then I tried to add one more memory about a friend's name via chat and it did not work sometimes. The "Updated saved memory" command never ran.
• I tried a few ways to ask it to run that tool call, update the saved memory, set a memory etc. It never could again or would say it didn't have the tools (which of course it does)
• I also asked the chat questions like "what is the key and value for XX memory?" and it would always get the key name wrong even thought it gets the actual memory correct. Very odd! Oddly when I tell it to "add a new memory with key = XX and value = XYZ" it actually does correctly assign the key. So it knows the difference.
• PS: I have a very high context window set in LM Studio and am using a great new model Mistral 24B
  that works fairly well!

Version Information

ghcr.io/danny-avila/librechat-dev latest d7fc65a29a47 2 days ago 1.93GB
ghcr.io/danny-avila/librechat-rag-api-dev latest 0c36e518c358 2 weeks ago 4.16GB

What browsers are you seeing the problem on?

Chrome (via Brave)

Relevant log output

I can provide specific logs if you need

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[onestardao]

ha — yep, memory bugs like these are why I ended up building a whole semantic firewall 🙃

You're likely hitting what we call an Interpretation Collapse — tool calls are passing values, but the model's semantic understanding of “what counts as a memory update” is inconsistent.
Basically: the LLM thinks it updated the memory, but forgot to tell the GUI. Or it told the GUI something almost right, and everything desyncs.

And that classic "add a new memory with key=X and value=Y" trick?
→ That’s bluffing. It often says it worked, but didn’t actually patch the memory registry.
(Likely hallucinated a success confirmation based on pattern, not actual backend state.)

---

We're tackling similar bugs over at WFGY Problem Map
Relevant issues that match what you're seeing:

• #2 Interpretation Collapse
• #4 Bluffing / Overconfidence
• #7 Memory Breaks Across Sessions

✅ We added ΔS consistency checks to verify what the model believes it just did, versus what actually happened.
Might be useful if LibreChat wants to detect ghost memory edits, or trigger a sanity check post-tool-call.

Let me know if you want a reproducible test patch — already hit this wall hard, happy to spare others the head trauma 🫠

[jhmonroe]

@onestardao thanks for validating what i was experiencing.
i think some of the problem might be a slight delay when turning the memory toggle on and off. maybe i need to refresh browser a couple times every time i change this in testing. there's no ajax-style feedback when toggled. when the chat is updating the memory (correctly) there is no indication until it's complete. so sometimes just waiting a second helps

this was a successful instance:

[onestardao]

ha, yep — that’s the ghost memory spiral we keep running into 😅
Looks fine, confirms update, even replays it back... but surprise: the registry still thinks it’s 2022 and loves hiking.

It’s like the model gets emotionally attached to a hallucinated update and refuses to actually commit it
We ended up adding ΔS-level semantic tension checks just to catch stuff like this mid-collapse.

Anyway, super appreciate you confirming — this kind of memory weirdness tends to gaslight even experienced devs.
Let me know if you want to cross-compare behavior across engines. I’ve seen identical failures trigger in different ways depending on whether the model is "trying to impress" the tool caller or not.

[danny-avila]

The main issue with the current memory implementation is that it runs on every request, so the "memory agent" LLM is always tempted to perform an action (some LLMs err on the side of calling a tool if it has access to tools, despite the user's best prompting against such actions).

Note, the "memory agent" runs separately from the main chat LLM.

This was already observed on initial implementation and the goal is to introduce a way to only run the "memory agent" when the user makes a clear intention for the system to "remember" or "forget" something, so we can both:

• optimize requests, as most will not require memory actions, as intended usage is only remember/forget when the user requests such actions
• avoid LLM erratic behavior, when it errs on remembering senseless/confusing information, or when not instructed to.

So we will add a "semantic check" like @onestardao may have already done in their fork, but also make this configurable, so that the admin can customize the phrases to denote semantic similarities to.

[onestardao]

Totally makes sense — and big thanks for the thoughtful response.

You're spot on that the memory agent shouldn't run on every turn — that itself causes a semantic reverberation where the model tries to act on “ghost traces” from previous hallucinated states.

For the semantic check, we ended up using a pretty lightweight approach:

Measure ΔS = 1 - cos(embedding angle) between memory state vectors

Add a logic vector λ_observe (→ convergent | ← divergent | <> recursive | × chaotic)

If ΔS > 0.6 or logic jumps, trigger BBCR (collapse–rebirth) fallback

It’s super cheap to compute, but gives a decent signal when the model is pretending to update memory but actually refuses to overwrite its internal state.

We noticed it mostly happens when:

Memory is injected with tool-caller context (vs. user-intent context)

The model gets “attached” to an old hallucinated state (even after replay)

Happy to share the ΔS config snippet we use, or if you want to cross-test in your own fork. Either way, I’m just glad someone else sees this issue — it’s haunted us for months 😅