PostgreSQL metadata filtering

[LeandroVidal555]

I have a postgres table which I've been populating with embeddings using Document Store.

The chatflow just retrieves info from that table using a Conversational Retrieval QA chain and is thus very simple:

I would like to filter the response info based on the documents' metadata in real time. Filter criteria should be sent via the chat API (or whatever method is more adequate) and the chat should respond exlusively based on the filtered documents. Is there a best way to achieve this on Flowise? Every idea I had so far incurred in concurrency and chat history collisions.

[LeandroVidal555]

Solved: noticed that I could introduce a metadata tag with the Doc Store and use a variable in the pg node filter. Then, using the API, the client introduces the metadata tag in the request body.

[prakashsonti]

Was this using Document Store or actually Vector Store? I am trying to do the same to add a dynamic filter like this pgMetadataFilter:"{"user_id":"$flow.state.user_id"}"

However, $flow.state.user_id does not appear to be getting translated.

Please advise how you were able to solve it.

Thanks

[prakashsonti]

I am able to filter by hardcoding a value. This is telling me that metadata filter is working. The question still remains, how do I pass this value dynamically in the new Agent where Vector database is configured and there is a setting for pgMetaDataFilter.

[LeandroVidal555]

Well this was quite long ago and I don't remember very well, but to answer your latest question: This is not an agent, but a ChatFlow. And I used Postgres with pgvector as a Vector Database.

To pass a pgMetaDataFilter value dynamically, which means setting the value when the query is made to the chatflow, you will have to go to your ChatFlow and Settings -> Configuration -> Override Configuration -> Turn on "Enable Override Configuration" -> "Postgres" -> Turn on "pgMetadataFilter"

After this, you should be able to make a query with a body like this:
{ "question": "Hello", "overrideConfig": { "pgMetadataFilter": { "your_key": your_value } } }

Is this what you were asking for?

[prakashsonti]

@LeandroVidal555 Thanks for your quick response. I v1 chatflow, I was able to find the setting you mentioned and turned it on. I have a website through which I call the chatbot. In the ChatbotinitFull() config, I added a section of overrideConfig:
overrideConfig: {
vars: {
user_id: userId, // Override the user_id variable
foo: "bar"
},
pgMetadataFilter: { user_id: userId }
},

I am not seeing results getting filtered by the metadata column called user_id. I also turned LangSmith Chatflow on and don't see this getting passed to the Postgres pgvector database. Am I mising something? Hope what I said makes sense.

As it relates to my original question: In AgentFlow version 2, the new chatflow, I tried looking for the same setting but could not find it. In AgentNode configuration, I added pgVector DB and it has a setting as shown in screenshot. I understand there is a BUG that is open for this.

I am trying to move to V2 agent flow but am not picky. Any suggestions you may offer would be helpful. Thanks