|
| 1 | +Here’s a brief README for the `memory` directory: |
| 2 | +# Memory Management with LlamaIndex and Perplexity Sonar API |
| 3 | + |
| 4 | +## Overview |
| 5 | +This directory explores solutions for preserving conversational memory in applications powered by large language models (LLMs). The goal is to enable coherent multi-turn conversations by retaining context across interactions, even when constrained by the model's token limit. |
| 6 | + |
| 7 | +## Problem Statement |
| 8 | + |
| 9 | +LLMs have a limited context window, making it challenging to maintain long-term conversational memory. Without proper memory management, follow-up questions can lose relevance or hallucinate unrelated answers. |
| 10 | + |
| 11 | +## Approaches |
| 12 | +Using LlamaIndex, we implemented two distinct strategies for solving this problem: |
| 13 | + |
| 14 | +### 1. **Chat Summary Memory Buffer** |
| 15 | +- **Goal**: Summarize older messages to fit within the token limit while retaining key context. |
| 16 | +- **Approach**: |
| 17 | + - Uses LlamaIndex's `ChatSummaryMemoryBuffer` to truncate and summarize conversation history dynamically. |
| 18 | + - Ensures that key details from earlier interactions are preserved in a compact form. |
| 19 | +- **Use Case**: Ideal for short-term conversations where memory efficiency is critical. |
| 20 | + |
| 21 | +### 2. **Persistent Memory with LanceDB** |
| 22 | +- **Goal**: Enable long-term memory persistence across sessions. |
| 23 | +- **Approach**: |
| 24 | + - Stores conversation history as vector embeddings in LanceDB. |
| 25 | + - Retrieves relevant historical context using semantic search and metadata filters. |
| 26 | + - Integrates Perplexity's Sonar API for generating responses based on retrieved context. |
| 27 | +- **Use Case**: Suitable for applications requiring long-term memory retention and contextual recall. |
| 28 | + |
| 29 | +## Directory Structure |
| 30 | +``` |
| 31 | +memory/ |
| 32 | +├── chat_summary_memory_buffer/ # Implementation of summarization-based memory |
| 33 | +├── chat_with_persistence/ # Implementation of persistent memory with LanceDB |
| 34 | +``` |
| 35 | + |
| 36 | +## Getting Started |
| 37 | +1. Clone the repository: |
| 38 | + ```bash |
| 39 | + git clone https://github.com/your-repo/api-cookbook.git |
| 40 | + cd api-cookbook/perplexity-llamaindex/memory |
| 41 | + ``` |
| 42 | +2. Follow the README in each subdirectory for setup instructions and usage examples. |
| 43 | + |
| 44 | +## Contributions |
| 45 | + |
| 46 | +If you have found another way to do tackle the same issue using LlamaIndex please feel free to open a PR! Check out our `CONTRIBUTING.md` file for more guidance. |
| 47 | + |
| 48 | +--- |
| 49 | + |
0 commit comments