Create graph "properties" (using LLMGraphTransformer.convert_to_graph_documents)

[pierreoberholzer]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

Add the notion of properties to the nodes and relationships generated by the LLMGraphTransformer.convert_to_graph_documents method.

Motivation

The process_response method inLLMGraphTransformer.convert_to_graph_documents is very slow in case a complex JSON is passed as an input (e.g. 60-90s). It seems that passing allowed_nodes and allowed_relationships helps, but with the major issue of losing content.

Proposal (If applicable)

The proposal is adjust the output model so that ALL content is mapped under nodes/relationships properties. This would smoothly integrate with Neo4j for example, which supports properties.

FYI @tomasonjo

[tomasonjo]

I fail to see how adding properties would speed up the extraction

[pierreoberholzer]

Good to hear. The use case I'm in would ask for an hybrid extraction : upfront schema + holistic population. I've tried the full graph generation mode along Constructing knowledge graphs from text using OpenAI functions. It works fine for one JSON record, but does not manage to absorb 500 JSON records - still running after 6h of execution (!).

[pierreoberholzer]

In short - The LLM's understanding would be : "Before I dive to deep into the complex structure of your JSONs, help me with some rudimentary domain knowledge, and I'll add the rest as properties". Of course if the process would be fast enough, happy to use the full graph.

[pierreoberholzer]

I have tried those models on my use case..

• gpt-4-0125-preview : Graph: high quality (granular and correct), QA: 3/3 correct answers, Time: 42s per record
• gpt-3.5-turbo-0125 : Graph: low quality (missing content), QA: 2/3 correct answers, Time: 16s per record
• llama3: Graph: low quality (missing content, wrong entity type), QA: 1/3 correct answers, Time : 11s per record

Sure you will perform more thorough benchmarking soon.

[danielosagie]

Hey thank you for you work. Would it be possible to share the code you wrote for using llama? I am trying out using llama 3: 70B with it and it would be nice to see how you wrote it

[pierreoberholzer]

Hi @danielosagie,

Just take @tomasonjo's code as base [1] and use the following..

from langchain_groq import ChatGroq

GROQ_API_KEY = "gsk_..."
os.environ["GROQ_API_KEY"] = GROQ_API_KEY 
...
llm = ChatGroq(temperature=0, model_name="llama3-70b-8192")

[1] https://medium.com/neo4j/enhancing-the-accuracy-of-rag-applications-with-knowledge-graphs-ad5e2ffab663