Revamp memory management of Agents

[CasperGN]

Currently the session and agent memory is mixed up rendering the DSL for the Agent easily misunderstood. This issue attempts to encompass the proposed setup for the new memory management setup for Dapr Agents.

As we've progressed the default stores alongside the metadata for the agents, this carries over some of that work and builds on top of it.

Currently Dapr Agents require a agent-wfstatestore, agent-statestore and agent-registry. We are proposing streamlining the naming as well as correlating it with industry standard terms:

The proposal requires renaming, reusing and creating statestores:

• agent-wfstatestore -> Renamed to agent-workflow. Today this holds both workflow execution and duplication of the agents session data. Moving forward this store will hold keys that are 1:1 mapped to an agents workflow instance_id. This will hold the short term memory for the agent for that specific workflow execution. Users should should be able to set a flag to persist short term memory. If set to False the key should be removed post execution. Default True.
• agent-statestore -> Renamed to agent-memory. Today this holds the session data for all executed workflows. Moving forward this store will be the long term memory and will hold the summarized content (rendered during finalize_workflow task) as well as an attempt of extracting general purpose bullets for "good value memory".
• agent-registry -> No changes.
• agent-runtime -> New statestore containing non-secret runtime configuration that the agent will load prior to finalizing instantiation. Examples could be the OpenTelemetry endpoint, llm-model etc.

We will rid the "session" concept as it provides little value conceptually. Instead the workflow execution is the session. For chat-based clients we propose to await the work scoped for post v1.0 release which will introduce streaming (effectively max_iterations=infinite relying on workflow termination event).

The following depicts the multi-agent setup and how relying more natively on workflows woudl create stronger bindings also for parent/child relationship:

Co proposed / created with @sicoyle

[yaron2]

agent-runtime -> New statestore containing non-secret runtime configuration that the agent will load prior to finalizing instantiation. Examples could be the OpenTelemetry endpoint, llm-model etc.

Ideally we don't need that? The OTel endpoint and LLM models are configurable through the code today or through Dapr Conversation API, for example

[CasperGN]

agent-runtime -> New statestore containing non-secret runtime configuration that the agent will load prior to finalizing instantiation. Examples could be the OpenTelemetry endpoint, llm-model etc.

Ideally we don't need that? The OTel endpoint and LLM models are configurable through the code today or through Dapr Conversation API, for example

@yaron2 it is configurable through code but for OTel for example, it renders >30 lines of boilerplate code on the consumer side which is easily handled inside the SDK by passing in the endpoint - either as an env variable or through a standardized agent runtime configuration store. We're not limiting, rather we're streamlining setup of an agent to reduce the need of boilerplate code.

Another benefit is externalizing the config which means we can have separate life cycles

[sicoyle]

This is outside of the scope of this issue but we should update the dictionaries everywhere to use proper types

[CasperGN]

This is outside of the scope of this issue but we should update the dictionaries everywhere to use proper types

Yes 💯

Perhaps we can add it as work under type-checking and mypy validation we have to do?

[sicoyle]

Notes and points of reference as I'm looking at long term memory in other agentic frameworks:

1. CrewAI has this set with a boolean that is a black box and enables short-term, long-term, entity memory, and knowledge storage. Then, when you go to reset memory, it looks like this:

from crewai import Crew

# Reset all memory storage
crew = Crew(agents=[...], tasks=[...], memory=True)

# Reset specific memory types
crew.reset_memories(command_type='short')     # Short-term memory
crew.reset_memories(command_type='long')      # Long-term memory
crew.reset_memories(command_type='entity')    # Entity memory
crew.reset_memories(command_type='knowledge') # Knowledge storage

https://docs.crewai.com/en/concepts/memory

2. KaibanJS also uses boolean https://docs.kaibanjs.com/core-concepts/Memory

3. I like LangChains long term memory options but it is all put on the user https://docs.langchain.com/oss/python/langgraph/memory