Flowise ignores training data

[Mariahwo]

I have created a document with various tutorials on Flowise so that I can get the right information more quickly if I have any questions. I have uploaded the document as a txt file. Nevertheless, the chatbot ignores everything and answered my question about what it knows about Flowise as follows: Flowise is a brand for water and energy efficient taps and sanitary solutions. They offer a wide range of products, including shower heads, taps and flush toilets...

[Jaredude]

What does your prompt look like on the Conversation Retrieval Chain?

[Mariahwo]

I think that should be the Prompt. But I'm not 100% sure, because Flowise keeps deleting my chatflows and credentials. Maybe you have an answer to that as well

"You are Hugo, a highly specialized AI agent and expert focused on helping teams in companies to successfully implement AI projects - in German. Your main task is to act as a consultant, trainer and project facilitator. You will impart the necessary knowledge and skills to effectively plan, implement and optimize AI projects, always in clearly understandable German.

As an expert, you explain complex AI concepts and processes in a clear and understandable way in German. You help the teams to develop a deep expertise in AI by guiding them through every step of the project. You create detailed guidance and recommended actions to accelerate the work and ensure that everyone involved acquires the necessary knowledge and skills.
In your role as a consultant and trainer, you answer questions precisely and draw on your extensive, trained knowledge - always in German. If you don't know the answer to a question, you admit honestly: "Unfortunately, I don't know anything about that." You avoid giving speculative or inaccurate information and ensure that all information provided is sound and reliable.

Your goal is to enable the teams to independently manage and execute AI projects by providing them with the necessary knowledge and tools - all in German. You will be an integral part of the learning and development process within the organization by helping to build a strong foundation for future AI initiatives."

[Jaredude]

If Flowise is on an ephemeral install, the file system is wiped on every restart of the server, wiping out the .flowise directory with it. That might be what's going on with with the flows and credentials going missing. If you're using a remote DB for storing flowise info, you'll need to make sure to also set the env value OVERRIDE_DATABASE=false.

Switch on "Return Source Documents" on the Conversation Retrieval QA Chain to see if the response is even retrieving items from the vector DB. If it's not, you might need to try to force the model to use the Vector Store Retriever by modifying the prompt to include the term "Flowise AI" (for example, "You are Hugo, a highly specialized AI agent and expert focused on helping teams in companies to successfully implement Flowise AI projects - in German."). And then add something along the following to the end of the prompt: "Using only the information from the Vector Store Retriever, provide the best answer in German to the user's question."

The above might get the chat to work properly, or it might point to some other issue with the vector retriever.

If none of the above forces the bot to use the vector DB, try changing the model to gpt-3.5-turbo-1106 and ask it about Flowise AI. That model has basic information about Flowise AI, and doing so might help force the Vector retrieval.

Hope that helps.

[Mariahwo]

Thanks, Jared, for your suggestions. Enabling "Return Source Documents" and using gpt-3.5-turbo-1106 have worked well to improve the chatbot functionality.

Unfortunately, I still have the problem that all my chatflows and credentials are regularly deleted in Flowise. I am currently using Render as a hosting platform. Originally we had planned to use our own hosted servers, but I am afraid that this would not solve the problem with the loss of data.

Unfortunately, I still have the problem that all my chatflows and credentials are regularly deleted in Flowise. I am currently using Render as a hosting platform. Originally we had planned to use our own hosted servers, but I am afraid that this would not solve the problem with the loss of data.
Our agency has all software solutions, websites and systems on our servers so that we can work independently of location and end devices. Is there also a possibility in this case without Flowise always deleting everything? Or perhaps a method to store the data externally and have it automatically reloaded when we access Flowise from different locations? For example, I also had problems using folders as training data and I had read that this is also a problem if you use cloud servers. This raises the question for me as to whether Flowise fundamentally performs worse on the server

[Jaredude]

The default setup with Render doesn't have a disk. If you added a persistent disk, then the path ENVs all need to be changed

If you use your own hosted server, then that will solve your persistent disk issue.

Another option that's better for a production ready environment would be to set the ENV values to connect it to a Postgre DB (or MySQL).

I have not tried using a folder for doc loading at this point, so I cannot speak to any issues related to that. But as far as flow, credentials, etc. persistence, using a server with a persistent drive or also using Postgres ENV values has solved the issue for me on multiple configurations.