feat: add Codespace support and libssl3 dependency setup for easy install (automatically install Sidecar binary-dependent libraries for Mac and Linux)

README.md

@@ -3,7 +3,7 @@
     <h1>SOTA SWE</h1>
 </div>
 
-**SotaSWE is a VSCode extension built atop [the leading agentic framework](https://github.com/codestoryai/sidecar) on SWE-bench Lite.**
+**SotaSWE is a VSCode extension built atop [the leading agentic framework](https://github.com/codestoryai/sidecar) on SWE-bench Lite. This is a working experimental repository for Memory LLM Agents using Pydantic to simulate human learning and thought and for aider using pydantic and custom tool support. I also plan on creating a Frontend using Roo-Cline and combining its features with the SOTA SWE features as well as the above. This build may be unstable so use with caution.**
 
 ![Latest release](https://img.shields.io/github/v/release/codestoryai/extension?label=version)
 ![Discord Shield](https://discord.com/api/guilds/1138070673756004464/widget.png?style=shield)
@@ -32,10 +32,47 @@ There are many ways in which you can participate in this project, for example:
 
 If you are interested in fixing issues and contributing directly to the code base,
 
+### System Requirements
+
+The extension requires the following system dependencies:
+- libssl3 (OpenSSL 3.0)
+- Python 3.9 or newer
+- pydantic v2
+
+These dependencies will be automatically installed when running in GitHub Codespaces. For local development:
+
+#### Ubuntu/Debian
+```shell
+# Add Ubuntu 22.04 repository for libssl3
+echo "deb http://archive.ubuntu.com/ubuntu jammy main" | sudo tee /etc/apt/sources.list.d/jammy.list
+sudo apt-get update
+sudo apt-get install -y libssl3 python3.9 python3-pip
+python3 -m pip install --upgrade pip
+python3 -m pip install -U pydantic
+```
+
+#### macOS
+```shell
+brew install openssl@3
+brew install [email protected]
+python3 -m pip install --upgrade pip
+python3 -m pip install -U pydantic
+```
+
+#### Windows
+The required OpenSSL libraries are bundled with the Windows binary.
+```shell
+# Install Python 3.9 or newer using Windows Store or from python.org
+python3 -m pip install --upgrade pip
+python3 -m pip install -U pydantic
+```
+
+### Development Setup
+
 1. Install dependencies
 
 ```shell
-  npm install
+npm install
 ```
 
 2. Run the extension in development mode with hot-reload

agentic-memory/README.md

@@ -0,0 +1,18 @@
+# Agent Memory - Can LLMs *Really* Think?
+
+<img src="./media/memory.png" width=600>
+
+*[Cognitive Architectures for Language Agents, 2024](https://arxiv.org/pdf/2309.02427)*
+
+LLMs are considered "stateless" in that every time you invoke an LLM call, it is like the first time it's ever seen the input being passed through. Given this quirk, multi-turn LLM agents have a unique challenge to overcome with fully understanding and navigating a vast world model which we humans do naturally.
+
+Being a human has a lot of advantages over a language model when executing a task. We bring our general knowledge about the world and lived experience, our understanding of prior similar task experiences and their takeaways, what we've specifically learned how to do or been taught, and then our ability to instantly contextualize and shape our approach to a task as we're working through it. In essence, we have advanced memory and the ability to learn from and apply learnings to new experiences. 
+
+LLMs sort of have some memory, mostly their general knowledge or traits picked up from training and additional fine tuning but suffer from a lack of the other characteristics outlined prior. To compensate for this, we can model different forms of memory, recall, and learning within our agentic system design. Specifically, we'll create a simple RAG agent to model 4 kinds of memory:
+
+- **Working Memory** - Current conversation and immediate context
+- **Episodic Memory** - Historical experiences and their takeaways
+- **Semantic Memory** - Knowledge context and factual grounding
+- **Procedural Memory** - The "rules" and "skills" for interaction
+
+These four memory systems provide a holistic approach to understanding and architecting a part of cognitive design into an agent application. In this notebook we'll break down each type of memory and an example approach to implementing them into a whole agent experience.

agentic-memory/agentic_memory.py

@@ -0,0 +1,197 @@
+from datetime import datetime
+from typing import List, Optional, Set
+from pydantic import BaseModel, Field
+from langchain_openai import ChatOpenAI
+from langchain_core.messages import HumanMessage, SystemMessage, AIMessage
+from langchain_core.prompts import ChatPromptTemplate
+from langchain_core.output_parsers import JsonOutputParser
+import weaviate
+
+# Base Message Model
+class Message(BaseModel):
+    """Base message model for all communications"""
+    role: str
+    content: str
+    timestamp: datetime = Field(default_factory=datetime.now)
+
+# Memory Models
+class WorkingMemory(BaseModel):
+    """Stores current conversation context and active state"""
+    messages: List[Message] = []
+    system_prompt: str = "You are a helpful AI Assistant."
+    semantic_context: Optional[str] = None
+
+class EpisodicMemory(BaseModel):
+    """Stores historical experiences and reflections"""
+    conversation: str
+    context_tags: List[str]
+    conversation_summary: str
+    what_worked: str
+    what_to_avoid: str
+
+class SemanticMemory(BaseModel):
+    """Stores factual knowledge and information"""
+    chunk: str
+
+class ProceduralMemory(BaseModel):
+    """Stores interaction guidelines and learned behaviors"""
+    guidelines: List[str]
+
+# Memory Tools
+class MemoryTools(BaseModel):
+    """Tools for memory operations"""
+
+    def format_conversation(self, messages: List[Message]) -> str:
+        """Format messages into a readable conversation string"""
+        conversation = []
+        for message in messages[1:]:  # Skip system message
+            conversation.append(f"{message.role.upper()}: {message.content}")
+        return "\n".join(conversation)
+
+    def episodic_recall(self, query: str, vdb_client) -> EpisodicMemory:
+        """Retrieve relevant episodic memory"""
+        episodic_memory = vdb_client.collections.get("episodic_memory")
+        memory = episodic_memory.query.hybrid(
+            query=query,
+            alpha=0.5,
+            limit=1,
+        )
+        props = memory.objects[0].properties
+        return EpisodicMemory(
+            conversation=props['conversation'],
+            context_tags=props['context_tags'],
+            conversation_summary=props['conversation_summary'],
+            what_worked=props['what_worked'],
+            what_to_avoid=props['what_to_avoid']
+        )
+
+    def semantic_recall(self, query: str, vdb_client) -> str:
+        """Retrieve relevant semantic knowledge"""
+        coala_collection = vdb_client.collections.get("CoALA_Paper")
+        memories = coala_collection.query.hybrid(
+            query=query,
+            alpha=0.5,
+            limit=15,
+        )
+        combined_text = ""
+        for i, memory in enumerate(memories.objects):
+            combined_text += f"\nCHUNK {i+1}:\n"
+            combined_text += memory.properties['chunk'].strip()
+        return combined_text
+
+    def load_procedural_memory(self) -> ProceduralMemory:
+        """Load procedural memory guidelines"""
+        with open("./procedural_memory.txt", "r") as content:
+            guidelines = content.read().split("\n")
+        return ProceduralMemory(guidelines=guidelines)
+
+# Memory Agent
+class MemoryAgent(BaseModel):
+    """Main agent class integrating all memory types"""
+    working_memory: WorkingMemory = Field(default_factory=WorkingMemory)
+    tools: MemoryTools = Field(default_factory=MemoryTools)
+    llm: ChatOpenAI = Field(default_factory=lambda: ChatOpenAI(temperature=0.7, model="gpt-4"))
+    vdb_client: Optional[object] = None
+
+    def initialize(self, vdb_client):
+        """Initialize the agent with vector database client"""
+        self.vdb_client = vdb_client
+
+    def update_system_prompt(self, query: str) -> str:
+        """Update system prompt with memory context"""
+        # Get episodic memory
+        episodic = self.tools.episodic_recall(query, self.vdb_client)
+
+        # Load procedural memory
+        procedural = self.tools.load_procedural_memory()
+
+        # Format system prompt
+        prompt = f"""You are a helpful AI Assistant. Answer the user's questions to the best of your ability.
+        You recall similar conversations with the user, here are the details:
+
+        Current Conversation Match: {episodic.conversation}
+        What has worked well: {episodic.what_worked}
+        What to avoid: {episodic.what_to_avoid}
+
+        Use these memories as context for your response to the user.
+
+        Additionally, here are guidelines for interactions with the current user:
+        {' '.join(procedural.guidelines)}"""
+
+        return prompt
+
+    def get_semantic_context(self, query: str) -> str:
+        """Get relevant semantic context"""
+        context = self.tools.semantic_recall(query, self.vdb_client)
+        return f"""If needed, Use this grounded context to factually answer the next question.
+        Let me know if you do not have enough information or context to answer a question.
+
+        {context}
+        """
+
+    def process_message(self, user_input: str) -> str:
+        """Process user message and generate response"""
+        # Update system prompt
+        system_prompt = self.update_system_prompt(user_input)
+        system_message = SystemMessage(content=system_prompt)
+
+        # Get semantic context
+        semantic_context = self.get_semantic_context(user_input)
+        semantic_message = HumanMessage(content=semantic_context)
+
+        # Create user message
+        user_message = HumanMessage(content=user_input)
+
+        # Update working memory
+        self.working_memory.messages = [
+            system_message,
+            *[msg for msg in self.working_memory.messages if not isinstance(msg, SystemMessage)],
+            semantic_message,
+            user_message
+        ]
+
+        # Generate response
+        response = self.llm.invoke(self.working_memory.messages)
+
+        # Add response to working memory
+        self.working_memory.messages.append(response)
+
+        return response.content
+
+    def save_episodic_memory(self):
+        """Save conversation to episodic memory"""
+        conversation = self.tools.format_conversation(self.working_memory.messages)
+
+        # Create reflection using LLM
+        reflection_prompt = ChatPromptTemplate.from_template("""
+        You are analyzing conversations to create memories that will help guide future interactions.
+        Review the conversation and create a memory reflection following these rules:
+        1. For any field where you don't have enough information, use "N/A"
+        2. Be extremely concise - each string should be one clear, actionable sentence
+        3. Focus only on information that would be useful for future conversations
+        4. Context_tags should be specific enough to match similar situations but general enough to be reusable
+
+        Output valid JSON in exactly this format:
+        {
+            "context_tags": [string],
+            "conversation_summary": string,
+            "what_worked": string,
+            "what_to_avoid": string
+        }
+
+        Here is the conversation:
+        {conversation}
+        """)
+
+        reflection = reflection_prompt | self.llm | JsonOutputParser()
+        memory = reflection.invoke({"conversation": conversation})
+
+        # Save to vector database
+        episodic_memory = self.vdb_client.collections.get("episodic_memory")
+        episodic_memory.data.insert({
+            "conversation": conversation,
+            "context_tags": memory['context_tags'],
+            "conversation_summary": memory['conversation_summary'],
+            "what_worked": memory['what_worked'],
+            "what_to_avoid": memory['what_to_avoid'],
+        })