-
-
Notifications
You must be signed in to change notification settings - Fork 52
Description
Describe the bug
The base Memory class in mesa-llm uses a dictionary-based staging area step_content that only stores one entry per event type. When multiple events of the same type occur during a single simulation step (e.g., an agent receiving messages from multiple neighbors), subsequent calls to add_to_memory overwrite previous ones, leading to permanent data loss.
Expected behavior
In line with discussions at #76
Communicative events (messages, actions, reasoning steps) should be additive. An agent receiving five messages in one step should have all five recorded.
Entries of type=observations may remain state-based (overwriting) to prevent LLM context bloat, but discrete events must be preserved.
To Reproduce
"""Sample test to show that step_content overwrites same-type entries"""
from unittest.mock import MagicMock
from mesa_llm.memory.st_memory import ShortTermMemory
agent = MagicMock()
agent.model.steps = 1
memory = ShortTermMemory(agent=agent, n=5)
# Simulate 3 agents sending messages to this agent in the same step
memory.add_to_memory("message", {"sender": "Agent_1", "msg": "Attack!"})
memory.add_to_memory("message", {"sender": "Agent_2", "msg": "Defend!"})
memory.add_to_memory("message", {"sender": "Agent_3", "msg": "Retreat!"})
print(f"Messages in step_content: {memory.step_content}")
# Output: {"message": {"sender": "Agent_3", "msg": "Retreat!"}}
# Agent_1 and Agent_2's messages are overwritten
# Proof: only 1 message key exists
assert list(memory.step_content.keys()).count("message") == 1
print(
f"\nOnly Agent_3's message survived. Agent_1 and Agent_2 lost.(length of step_content: {list(memory.step_content.keys()).count('message')})"
)
Additional context
The initial fix I had in mind was to transform the datatype of step_content to enable it to store additional lists for when multiple messages are sent during the same step.
But on further thought I think this would confuse the llm more and make it slower by supplying it a large amount of memory to parse through rather than providing it crisp info.
@sanika-n @wang-boyu @colinfrisch , What do you guys think about this?