Skip to content

Memory staging area overwrites concurrent events of same type (leads to Data Loss) #137

@psbuilds

Description

@psbuilds

Describe the bug

The base Memory class in mesa-llm uses a dictionary-based staging area step_content that only stores one entry per event type. When multiple events of the same type occur during a single simulation step (e.g., an agent receiving messages from multiple neighbors), subsequent calls to add_to_memory overwrite previous ones, leading to permanent data loss.

Expected behavior
In line with discussions at #76

Communicative events (messages, actions, reasoning steps) should be additive. An agent receiving five messages in one step should have all five recorded.
Entries of type=observations may remain state-based (overwriting) to prevent LLM context bloat, but discrete events must be preserved.

To Reproduce

"""Sample test to show that step_content overwrites same-type entries"""
from unittest.mock import MagicMock
from mesa_llm.memory.st_memory import ShortTermMemory

agent = MagicMock()
agent.model.steps = 1

memory = ShortTermMemory(agent=agent, n=5)

# Simulate 3 agents sending messages to this agent in the same step
memory.add_to_memory("message", {"sender": "Agent_1", "msg": "Attack!"})
memory.add_to_memory("message", {"sender": "Agent_2", "msg": "Defend!"})
memory.add_to_memory("message", {"sender": "Agent_3", "msg": "Retreat!"})

print(f"Messages in step_content: {memory.step_content}")
# Output: {"message": {"sender": "Agent_3", "msg": "Retreat!"}}
# Agent_1 and Agent_2's messages are overwritten 

# Proof: only 1 message key exists
assert list(memory.step_content.keys()).count("message") == 1
print(
    f"\nOnly Agent_3's message survived. Agent_1 and Agent_2 lost.(length of step_content: {list(memory.step_content.keys()).count('message')})"
)

Additional context
The initial fix I had in mind was to transform the datatype of step_content to enable it to store additional lists for when multiple messages are sent during the same step.

But on further thought I think this would confuse the llm more and make it slower by supplying it a large amount of memory to parse through rather than providing it crisp info.

@sanika-n @wang-boyu @colinfrisch , What do you guys think about this?

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugRelease notes label

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions