Best Practices for :memory: mode and Semantic Cache

[gururaser]

Hi,

I want to develop semantic cache using Qdrant according to this article and I have questions about some terms:

1- Can I use Quantization with :memory: mode? I want to save millions of vectors with their basic payload in memory with Binary Quantization
2- What about vector and payload indexing?

What are the best practices?

[generall]

No, :memory: is not designed for any significant load. Please prefer docker or cloud instead.

[generall]

https://github.com/qdrant/qdrant-client?tab=readme-ov-file#local-mode

[gururaser]

Article?

https://qdrant.tech/articles/semantic-cache-ai-data-retrieval/

[generall]

How is this article related to :memory: mode limitations?

[gururaser]

How is this article related to :memory: mode limitations?

In the article there is Google Colab link: https://colab.research.google.com/github/infoslack/qdrant-example/blob/main/semantic-cache.ipynb

In this notebook, it basically shows how to use Qdrant in :memory: mode in according to use for Semantic cache. But since it is a basic tutorial, the config settings of the Qdrant instance created in memory mode are quite limited. I was wondering what can be done about this

class SemanticCache:
    def __init__(self, threshold=0.35):
        self.encoder = TextEmbedding(model_name="BAAI/bge-small-en-v1.5")
        self.cache_client = QdrantClient(":memory:")
        self.cache_collection_name = "cache"

        self.cache_client.create_collection(
            collection_name=self.cache_collection_name,
            vectors_config=models.VectorParams(
                size=384,
                distance='Euclid'
            )
        )

        # Initialize Qdrant Client for external database
        self.db_client = QdrantClient(
            os.getenv("QDRANT_HOST"),
            api_key=os.getenv("QDRANT_API_KEY")
        )
        self.db_collection_name = os.getenv('QDRANT_COLLECTION_NAME')
        
        self.euclidean_threshold = threshold

[generall]

Simply don't use :memory: mode