Need some guidance about my requirement and some doubts

[sergiogcb]

Hi guys!

At the moment, I have the following requirement: I want to feed the OpenAI model 3.5 with the information we have from our apps in our documentation. This documentation is stored in a public Confluence space. Our idea is that this model is able to generate answers for customers in our service desk portal based in the information it has. It's basically really similar to what dosu bot is doing here.

What would be the best approach?

I'm currently going to develop it on Python, previously I tried with Typescript and using ApifyDataSetLoader. This was my previous approach:

`
const loader = new ApifyDatasetLoader("dataSetId", {
datasetMappingFunction: (item) =>
new Document({
pageContent: (item.text || "") as string,
metadata: { source: item.url },
}),
clientOptions: {
token: "tokenApify",
},
});

 const docs = await loader.load();

const vectorStore = await HNSWLib.fromDocuments(docs, new OpenAIEmbeddings());

const model = new OpenAI({
temperature: 0.2,

});

const chain = RetrievalQAChain.fromLLM(model, vectorStore.asRetriever(), {
returnSourceDocuments: true,
prompt: PromptTemplate.fromTemplate(
  "Prompt text" + "The ticket desciption is is: {query}"

)

});
const res = await chain.call({ query: "Question from customer" });

console.log(res.text);
console.log(res.sourceDocuments.map((d: Document) => d.metadata.source));`

• One of the doubts with this approach is that I'm always creating a new OpenAIEmbedding with this data, so I will always have the cost of generating the embedding when the data is always going to be the same. How could I do it so I can do the embedding once and then use that embedding I did? Otherwise, everytime the new code I do in Python, in case it's similar to this one, it will always have to pay the embedding price from OpenAI. Because we have nearly 800 pages in our Confluence, so it's going to be a big amount of tokens for the embedding. Should I have a class where I make the initial embedding and save it and then in a different class I load that embedding with one of the tools like Chroma, etc?
• In Python I'm going to use the ConfluenceLoader instead of the Apify one, the code was to give you an idea about what was my previous approach.
• Is the RetrievalQAChain the correct approach for this? Do you have another better solution?
• What could I do in order to keep training this model with the tickets where he answered? Or from tickets we've already have in our customer portal. Which Memory integration would you suggest?

I know it's a lot of questions, but I just want to be sure I take a good and safe approach for this requirement. I would like to know and be sure about what I'm going to do and the reasons in order to be optimized.

Thanks so much in advance!

Best regards,
Sergio

[dosubot[bot]]

🤖

Hi Sergio,

Your approach seems to be on the right track. Here are some suggestions based on your questions:

1. Embedding Generation: You're correct that generating embeddings for the same data repeatedly can be costly. One way to mitigate this is to generate the embeddings once and then save them for future use. You can use a tool like pickle in Python to serialize the embeddings and save them to a file. Then, you can load the embeddings from the file when you need them. Here's a simple example:

import pickle

# Save embeddings to a file
with open('embeddings.pkl', 'wb') as f:
    pickle.dump(embeddings, f)

# Load embeddings from a file
with open('embeddings.pkl', 'rb') as f:
    embeddings = pickle.load(f)

2. ConfluenceLoader: Yes, you can use the ConfluenceLoader in Python to load data from your Confluence space. The usage is similar to the ApifyDatasetLoader you used in TypeScript. You can refer to the ConfluenceLoader documentation for more details.

3. RetrievalQAChain: The RetrievalQAChain is a good choice for your use case. It uses a retrieval-based approach to answer questions, which is suitable for customer service scenarios where the answers are often based on existing information (in your case, the Confluence documentation). You can refer to the RetrievalQAChain documentation for more details.

4. Continuous Training: For continuous training, you can use the Memory class in LangChain to store past interactions and use them for training. You can save the tickets where the model answered and load them into the Memory when you train the model. Here's a simple example:

from libs.langchain.langchain.memory import Memory

# Initialize the memory
memory = Memory()

# Add a past interaction to the memory
memory.add_interaction(user_query, model_response)

# Use the memory for training
model.train(memory.get_interactions())

You can refer to the Memory documentation for more details.

I hope this helps! Let me know if you have any other questions.

Best,
Dosu

Sources

• libs/langchain/langchain/chat_models/openai.py
• libs/langchain/langchain/embeddings/openai.py
• libs/langchain/langchain/llms/openai.py
• libs/langchain/tests/unit_tests/document_loaders/test_confluence.py
• libs/langchain/langchain/document_loaders/confluence.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[sergiogcb]

@dosu-bot

What if I want to store that Memory instead of always having to create it?

This project is going to be stored in AWS, so the idea is that the Memory should be saved instead of having to create it all the time, just like you did in the example you gave me

[dosubot[bot]]

🤖

Hello,

Based on your requirements, you can extend the SimpleMemory class in LangChain to support saving and loading of data. Here's a simple way to do it:

import pickle
import boto3
from botocore.exceptions import NoCredentialsError

class AWSSimpleMemory(SimpleMemory):
    def __init__(self, bucket_name, file_name, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.s3 = boto3.client('s3')
        self.bucket_name = bucket_name
        self.file_name = file_name

    def save_data(self) -> None:
        """Save the memory data to a file in S3."""
        try:
            self.s3.put_object(Body=pickle.dumps(self.memories), Bucket=self.bucket_name, Key=self.file_name)
        except NoCredentialsError:
            print("No AWS credentials found")

    def load_data(self) -> None:
        """Load the memory data from a file in S3."""
        try:
            data = self.s3.get_object(Bucket=self.bucket_name, Key=self.file_name)
            self.memories = pickle.loads(data['Body'].read())
        except self.s3.exceptions.NoSuchKey:
            print(f"No object found in bucket {self.bucket_name} with key {self.file_name}")
        except NoCredentialsError:
            print("No AWS credentials found")

In this code, bucket_name is the name of your S3 bucket and file_name is the name of the file in the bucket where you want to save or load the memory data. The save_data method uses pickle.dumps to serialize the memory data and s3.put_object to write it to the file in S3. The load_data method uses s3.get_object to read the serialized data from the file in S3 and pickle.loads to deserialize it back into memory.

Please note that you need to have your AWS credentials configured correctly for this to work. You can do this by setting the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_REGION environment variables, or by using the AWS CLI or AWS SDKs to configure your credentials.

Also, please be aware that pickle is not secure against erroneous or maliciously constructed data, so you should only use it to load data that you have saved yourself.

I hope this helps! Let me know if you have any other questions.

Sources

• libs/langchain/langchain/memory/zep_memory.py
• libs/langchain/langchain/memory/simple.py
• libs/experimental/langchain_experimental/generative_agents/memory.py
• libs/langchain/langchain/schema/memory.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.