how to do filtering in pgvector

[deepak-hl]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

retriever = self.vector_store.as_retriever(search_kwargs={
        'k': limit, 
        'filter': {'portfolio_uri':{'$in': [params.portfolio_uri]}}
      })

Description

there is two table langchain pg_embeddings and langchain pg_collection, on which table we can apply filter?

System Info

langchain
pyschopg2

[dosubot[bot]]

In LangChain's PGVector integration, you can apply filters on both the pg_embeddings and pg_collection tables. The filtering operations are typically applied to the metadata fields of these tables.

• pg_embeddings Table: This table stores individual embeddings along with their associated documents and metadata. You can apply filters on the metadata fields using various operators like $eq, $ne, $in, etc. For example, you can filter based on a specific portfolio_uri as shown in your example code.

• pg_collection Table: This table is used to store collections of embeddings, and you can also apply filters on the metadata fields associated with collections.

The filtering options available include operators such as $eq, $ne, $lt, $lte, $gt, $gte, $in, $nin, $between, $like, $ilike, $and, and $or ^[1].

If you have specific metadata fields in either table that you want to filter by, you can use these operators to construct your filter queries.

To continue talking to Dosu, mention @dosu.

[deepak-hl]

@dosu how can i filter from langchain_pgembeddings from cmetada based on the source,
what is retreiver object for it?

[dosubot[bot]]

To filter from the langchain_pgembeddings table based on the source in the metadata, you can use SQL queries with a WHERE clause to specify the condition on the source field in the metadata. The retriever object for this setup is created using the VectorSQLDatabaseChainRetriever class. Here is the relevant code snippet:

retriever = VectorSQLDatabaseChainRetriever(
    sql_db_chain=chain, page_content_key="abstract"
)

This retriever is configured to work with the VectorSQLDatabaseChain and uses the page_content_key to specify the key for the content in the database ^[1].

To continue talking to Dosu, mention @dosu.