Does it make sense to pre-compute the compressions for each document chunk before indexing in a vector store?

[stephenleo]

Is it advisable to pre-compute every document chunk's compression before indexing it into a vector store?
This will allow us to run compression just once per document chunk instead of running it on every API call.

For example
Given N documents to index:

1. Split the document into chunks
2. Compute embedding for each original document chunks
3. Compress the text of the document chunks
4. Index the original document embedding vectors for most accurate search and the compressed text as metadata
5. Stuff the compressed texts into RAG prompt at runtime. No further compression at runtime

Please let me know if you forsee any issues with this approach?

[iofu728]

Hi @stephenleo, thanks for your support.

I think it's reasonable. However, one thing you might need to consider is that during retrieval, you may still need to use the original document, as the retrieval model is trained on natural language. After that, you can recall the compressed document and concatenate it to the prompt.