Totally lost: how to integrate some kind of RAG (on documents, and also past conversations)?

[alexispurslane]

I'm kind if just getting into LLMs, because they've finally gotten to the point now where I personally can see how they'd be reliably useful for me, and regarding RAG specifically, it sounds like something I'd want to use to give it long term memory of documentation and past conversations, but there's a bunch of ways I could see to do it and honestly I'm just lost. Does anyone here want to share their workflow so I can get some direction? I was considering using the pure query function of nano-graphrag as a tool for my LLM or something. Thanks!

[karthink]

Depending on your needs, this can be quite trivial. The simplest option is to use gptel-request -- you can read the buffer text, use it as a query for RAG, and call gptel-request on the concatenation of the RAG result and the original query.

#815 is an attempt to integrate it more closely into gptel's state machine. There are many approaches being considered. There are many ways to do this, but it's not clear yet what the most future-proof API for integrating RAG is.

[alexispurslane]

use it as a query for RAG

What tool would you recommend for actually handling this part of it? Like building the vector embedding database and querying it.

[karthink]

What tool would you recommend for actually handling this part of it? Like building the vector embedding database and querying it.

I'm not very knowledgeable about this aspect of the LLM space. I see new RAG solutions popping up every day, but I haven't tried any of them. It looks like a lot of work to set them up, and my free time is mostly consumed by gptel development.

So I can mostly help with integrating RAG software with gptel -- I am actively working on an interface for it right now.

John Wiegley has been working on a simple RAG client that you could try: https://github.com/jwiegley/rag-client

I plan to use this as the test case for RAG integration with gptel.